Case study: Stb-tester.com supports YouView’s launch on Sony Bravia TVs

18 December 2015

YouView, the on demand TV platform, is a joint venture bringing together some of the UK’s leading media businesses. The shareholders include BBC, ITV, Channel 4, Channel 5, BT, TalkTalk and Arqiva.

On 4th November 2015 YouView launched on a range of Sony Bravia TVs.

The biggest technical challenges of the project were in the integration, with frequent releases from YouView, Sony, MediaTek (the silicon vendor), and Google. Each release needed to be integrated and tested; the high pace of development —one or more releases every week— meant that manual testing alone wouldn’t be sufficient, and any automated testing needed to be reliable, flexible, and broad in coverage.

“Stb-tester.com’s automation of our test cases enabled the adoption of a Continuous Integration approach to development. This allowed us to spot defects and regressions much earlier, increasing our confidence in builds taken forward for release.”

The solution

YouView hired Stb-tester.com to develop automated test scripts and to provide testing services throughout the project.

Stb-tester.com implemented a broad range of automated testcases covering user journeys within the YouView User Interface, as well as journeys between YouView, the host Sony/Android UI, and third party apps. Testing services included:

• A Continuous Integration system that tested every change YouView developers made to the UI, and delivered the results directly to the developers, thus shortening the build-release-test cycle to less than 20 minutes. This catches defects earlier —when they are cheapest to fix— and it allowed YouView to reap the benefits of test automation from very early in the project.

“Stb-tester.com’s automated CI tests caught several regressions while we were refactoring in this Sprint.”

• A comprehensive functional acceptance report by the next day after every release.

• Uninterrupted soak-testing for days or weeks at a time, to find intermittent defects and gather extensive stability data.

After launch the test scripts were handed over to YouView and to Sony so that they can be used to test future software updates. All the test scripts were compatible with the stb-tester open-source software and with the stb-tester ONE appliance, which made handover painless thanks to the stb-tester ONE’s reliability, simplified management, and powerful triage & reporting tools.

“Stb-tester.com’s efforts to encapsulate the Continuous Integration system in docker containers has been a great help, making the handover fast and painless.”

Benefits of stb-tester

The benefits of test automation —scalability, reproducibility, and fast turn-around time— are well known. On top of that, stb-tester provided some unique advantages:

Stb-tester takes a “black box” approach, whereby testing is done in the same way that a real user interacts with the system: pressing buttons on a remote control and looking at the output on the screen. This is ideal for catching integration problems, unlike implementation-dependent “glass box” methods that target individual sub-systems.

Stb-tester’s “virtual-stb” technology allows testing in a virtualised environment, regardless of the implementation of the UI under test — be it an HTML5 app running in a browser, an Android app running in an emulator, or a desktop application framework such as Qt. This allowed the automated test scripts to be developed against YouView’s UI early in the project, even before real hardware was available. Later, the same test scripts were used to test against integrated builds on real hardware. This moved most of the test automation work off the critical path.

About YouView TV Ltd.

YouView, the on demand TV platform, is a joint venture bringing together some of the UK’s leading media businesses. The shareholders include BBC, ITV, Channel 4, Channel 5, BT, TalkTalk and Arqiva.

Launched in 2012, YouView has seen rapid growth in connected devices with over 2 million households across the UK enjoying the service.

YouView is available via retail, BT, TalkTalk and Plusnet as well as via Sony Bravia 2015 Television.

For more information visit: youview.com, @youview and facebook.com/youview.

About Stb-tester.com Ltd.

Stb-tester.com delivers best-in-class test automation tools and services to the set-top box, TV, and video-on-demand industries.

For more information please visit stb-tester.com or contact sales@stb-tester.com.