Celebrating 10 years of Stb-tester

How stb-tester works

Stb-tester captures the video from the HDMI output of your device under test (a set-top box, for example).

Stb-tester controls the device under test by sending infrared signals, pretending to be a human user pressing buttons on a remote control unit.

Stb-tester performs true black-box testing: Stb-tester’s interfaces to the system-under-test are the same as a real user (the remote control input and the video output). You can test your device or application just like real users do, and you can test the production version of your application – you don’t need instrumented debug builds.

Your test automation developers write testcases using the Python programming language. The testcase tells stb-tester when to send a command (such as pressing a button on the remote control) and what to expect to find on the screen:

  • An image of a particular button or menu element
  • A particular word or phrase (stb-tester can read text using OCR, optical character recognition)
  • Motion or the absence of motion (to test that video is playing or paused)
  • Presence or absence of black screen (for example to measure channel-change duration)

From these basic operations your test automation developers can build very advanced logic to automatically navigate menus, carry out user journeys, and perform end-to-end testing.

def test_that_pressing_EPG_opens_the_guide():
    press("KEY_EPG")
    wait_for_match(Guide.png)

Simple testcase

Stb-tester creates interactive reports that allow your QA team to focus on triaging failures.

The reports contain video recordings of each test run. You can configure stb-tester to gather logs from the device-under-test; these logs appear in the report under the relevant test run. You can also configure the report to display any data that you have collected; for example if you’re soak-testing a channel-change test, you might want the duration of the channel-change to appear as a column in the report, so that you can sort by that column and instantly find the outliers.

Stb-tester shines in functional testing, driving end-to-end user journeys and providing automation of User Interface testing where automation was not previously feasible;
in soak testing, running thousands of test iterations to reproduce very intermittent issues and to measure mean time to failure;
and in regression testing, reducing turn-around time and allowing your manual QA team to focus on what humans do best, leaving the tedious repetitive tasks to the computer.

Stb-tester accelerates your software development process. By adding stb-tester automated testing into your Continuous Integration pipeline, new defects can be detected early in the development life-cycle when it is cheapest to fix them. This helps to speed up the develop-test-release cycle, making your organisation more agile.