Benefits of stb-tester

True black-box testing of your production application

• End-to-end functional testing.
• No need for special instrumented builds.
• Stb-tester’s interfaces to the system-under-test are the same as a real user: The remote control input, and the video output.

Run soak tests around the clock

• Stb-tester has been successfully used to identify the presence and absence of defects with a reproducibility lower than 1% – well below the error rate of manual testing.
• To confidently distinguish between a 1% defect and a 2% defect you need over 1000 test runs.
• Stb-tester can run sufficient number of iterations to achieve statistically significant results, increasing confidence and removing uncertainty.
• Stb-tester’s reporting interface has been carefully designed to optimise the triaging of test results, so that in under 5 minutes you can find the few “interesting” test runs that you need to focus your analysis on, out of tens of thousands of test runs. (For an example see “Test failures vs. errors, and preconditions”.)
• Stb-tester can gather logs from the system under test. Stb-tester’s reporting interface shows you the relevant logs for each test run.

Detect regressions earlier in the development lifecycle

• Stb-tester can be driven from existing Continuous Integration systems, such that any new defects can be detected early in the development lifecycle when it is cheapest to fix them.
• This helps to speed up the develop-test-release cycle, making your organisation more agile.
• Because stb-tester is open source it is very flexible, without the artificial barriers to integration that can come with proprietary software.

Scale your testing to multiple platforms

• Once the test scripts are written, testing can happen 24/7.
• Test effort can be scaled just by adding more hardware.
• We have users who use exactly the same automated test scripts to test their app on multiple different platforms, thus saving on development and maintenance costs, and ensuring that their application behaves consistently on all devices.

Clearly communicate the nature of defects

• Stb-tester’s reports are web-based and include video recordings of each test run. You can link directly to the video from your defect-tracking system, share the video with your technology partners responsible for fixing the defect, or play the video at defect-review meetings.

Monitor availability of live services

• Use stb-tester as part of operations for measuring real-world service availability.
• Test against your live services with the real infrastructure your customers are using.
• Be alerted to service outages before your users notice.

Key features

• Emulate user actions via infrared remote control.
• Recognize specific images such as UI elements.
• Read text from the screen using Optical Character Recognition (OCR).
• Detect motion, lack of motion, black screen, color tint.
• Measure duration of channel change, application launch, and other events.
• Save a video recording and extensive logs for each test run.
• View test results in a modern, web-based user interface.