stb-tester : Blog

10 Apr 2018.

Test-script development workflow

When you are developing new test scripts (or modifying existing ones) we recommend this workflow: Read more...

26 Mar 2018.

Testing the Amazon Fire TV with Stb-tester

Stb-tester supports automated testing on the Amazon Fire TV and Fire TV Stick. Read more...

23 Mar 2018.

Announcing Stb-tester integration with Atlassian Bamboo CI Server

The Stb-tester Platform now supports Atlassian's Bamboo Server for Continuous Integration (CI). Many of our customers already use Bamboo for continuous integration, deployment, and delivery. Now, it will only take a few minutes to get your Stb-tester end-to-end tests running from your Bamboo jobs. Read more...

20 Mar 2018.

Cropping reference images in your browser

When you are capturing reference images for your test scripts, the Stb-tester Portal now allows you to crop the images directly in your browser. Read more...

06 Feb 2018.

Stb-tester v28 released: Multi-threading support, better OCR, better video recordings

Version 28 of the stb-tester core Python API is now available. Read more...

17 Jan 2018.

Announcing the Stb-tester Cloud Platform

Stb-tester’s cloud-based portal centralizes your test results and dashboards. It allows remote testing and live monitoring from anywhere in the world — your devices can be spread across multiple network locations, and your test team can be remote too. Read more...

27 Feb 2017.

Stb-tester ONE v27.4: Better test farm configuration, new test-results API, and remote Android ADB control

Software update v27.4 for the stb-tester ONE is now available. It includes a more convenient configuration system for large test farms, a new REST API for searching past test results, support for controlling Android devices without a USB connection, and more. Read more...

21 Feb 2017.

Testing video playback on mobile devices

There are several mature open source and proprietary tools for testing the user interface of mobile apps on Android and iOS devices, but there aren’t many solutions for internet service providers or video-on-demand streaming services who want to test video playback on real mobile devices to validate their end-to-end video delivery. In this article I’ll discuss various ways of getting video frames from a mobile device for automated testing, and the advantages and trade-offs of each approach. Read more...

23 Jan 2017.

Stb-tester ONE v27.3 released

Software update v27.3 for the stb-tester ONE is now available. It adds support for the Sony PlayStation 3 & 4, Android devices, and more! Read more...

05 Jan 2017.

Controlling the Sony PlayStation 4 for test automation

To test a PS4 app you need a mechanism to control the PS4 and a way to get feedback to check that your app is behaving as it should. Read more...

12 Dec 2016.

Announcing the stb-tester camera for TVs and mobile devices

The stb-tester camera allows you to test the user interface of devices that don’t have an HDMI output, such as Smart TVs and mobile devices. Read more...

28 Oct 2016.

Build your own serial-to-ethernet server with a Raspberry Pi

A Serial Terminal Server makes a serial connection available over the network, so that your application can access your devices without a direct serial connection. Typically this is used to establish a bi-directional connection between networked applications and older industrial equipment. In this article we will focus on a much simpler use case: Capturing serial logs, where we only need communication in one direction — a common use case when testing set-top boxes. Read more...

04 Oct 2016.

Stb-tester ONE v26.6 released

Software update v26.6 for the stb-tester ONE is now available. It contains a JPEG thumbnail alternative to the Flash video in the web interface, and a bugfix for inode exhaustion on the stb-tester ONE’s test-results partition. Read more...

20 Sep 2016.

Add visual verification to your Selenium tests with stb-tester

Selenium is a test automation tool for web applications. It tests your web app by looking at the DOM – you can check that your web page contains a particular element like a div or a button, identified by id or css selector or xpath expression; once the element is located, you can check that it contains a particular text or other elements. However you can’t really check what your web app looks like. If you have complex CSS it can be all too easy to introduce unintended defects in your app’s visual appearance. Read more...

08 Aug 2016.

Meet the stb-tester team at IBC

The stb-tester team will be at the IBC exhibition in Amsterdam 9-13 September 2016. Read more...

22 Jul 2016.

Stb-tester ONE v26.2 released

Software update v26.2 for the stb-tester ONE is now available, with new features for managing & monitoring large test farms. Read more...

13 Jul 2016.

Stb-tester v26 released

Stb-tester v26 is now available, with support for frame-accurate performance measurements. Read more...

05 Jul 2016.

How we test the stb-tester ONE: Video-capture latency & accuracy

Updated 17 May 2017 with figures for our new hardware’s accuracy. Read more...

29 Jun 2016.

Run your tests!

If you have already automated one or more testcases, run them! Ideally run them every time a developer commits a change to the system-under-test. If that isn’t possible, at least run the tests against nightly builds. Run! Your! Tests! Read more...

21 Jun 2016.

New tutorial: Test development workflow with stbt.FrameObject, stbt auto-selftest and stbt-docker

We have just published an extensive tutorial about our recommended workflow for implementing automated test scripts, using stbt.FrameObject to extract information from the screen, using stbt lint and stbt auto-selftest to test your scripts locally, and using stbt-docker to develop on Mac or Windows. Read more...

03 Jun 2016.

Stb-tester v25 released

Stb-tester v25 is now available, with features that significantly improve the maintenance cost of your test scripts: Read more...

27 May 2016.

Stb-tester ONE v24.8 released

Software update v24.8 for the stb-tester ONE is now available. It fixes a regression introduced in v24.7 that caused the stb-tester ONE’s old USB infrared transmitter to fail for signals longer than 125 milliseconds. Read more...

26 May 2016.

Stb-tester ONE v24.7 released, with support for new infrared hardware

Software update v24.7 for the stb-tester ONE is now available. Read more...

26 May 2016.

A post-mortem report on the stb-tester ONE's infrared transmitter reliability

This is a technical report on our investigation into an intermittent problem with the stb-tester ONE’s infrared transmitter. Read more...

06 May 2016.

New tutorial video: How to create a custom remote control image

How to create a custom remote control image for the stb-tester ONE’s “manual control” web interface: Read more...

22 Apr 2016.

Stb-tester ONE v24.5 released

Software update v24.5 for the stb-tester ONE is now available. This release delivers a minor UI improvement, and a bugfix that affects a small number of users. Read more...

29 Feb 2016.

New videos & tutorials

We have just published three videos showcasing the stb-tester ONE, including a detailed guide for measuring channel change times on a set-top box: Read more...

11 Feb 2016.

Stb-tester ONE v24.4 released

Software update v24.4 for the stb-tester ONE is now available, with the ability to run tests from any git branch, and several bugfixes. Read more...

20 Jan 2016.

Stb-tester ONE v24.3 released

Software update v24.3 for the stb-tester ONE is now available. This includes better feedback when you have errors in your infrared configuration file, and other small usability improvements & bugfixes. Read more...

14 Jan 2016.

Stb-tester v24 released

Stb-tester v24 is now available, with many small Python API improvements and fixes. Read more...

11 Jan 2016.

Stb-tester ONE v23.7 released

Software update v23.7 for the stb-tester ONE is now available. This includes a new test-results interface with much better search capabilities, plus some minor fixes & updates. Read more...

18 Dec 2015.

New case study: stb-tester.com supports YouView’s launch on Sony Bravia TVs

In November 2015 YouView launched on a range of Sony Bravia televisions across the UK. The biggest technical challenges of the project were in the integration, with frequent releases from YouView, Sony, MediaTek, and Google. Read more...

16 Dec 2015.

Stb-tester ONE v23.6 released

Software update v23.6 for the stb-tester ONE is now available. This is a bugfix release. Read more...

02 Dec 2015.

Stb-tester.com to help DTVKit with test automation solution

Stb-tester.com is pleased to announce that DTVKit will be using our stb-tester tool as part of their test automation strategy, and we will be providing our expertise and support as a Partner Member of DTVKit. Read more...

09 Nov 2015.

Stb-tester ONE v23.5 released

Software update v23.5 for the stb-tester ONE is now available. It has a technology preview of upcoming improvements to the results viewer, as well as bugfixes. Read more...

24 Sep 2015.

Improve black-box testing agility: automatic self-regression tests

In the previous blog post we discussed the frame object pattern and how it could lead to greater agility and lower maintenance costs thanks to improved self-testability. This blog post discusses a technique to get the maximum value from this testability at near-zero cost. Read more...

08 Sep 2015.

Improve black-box testing agility: meet the Frame Object pattern

In the previous blog post we discussed the Page Object pattern and how it could be used to improve the readability of your test scripts. In this blog post we introduce the Frame Object pattern, a specialisation of the Page Object pattern that can be especially effective in stb-tester’s style of black-box UI testing¹. Frame Objects can reduce the maintenance cost of your test-packs, particularly when the UI-under-test changes.

By “black-box UI testing” we mean that you’re testing what the user sees, by capturing a video stream of the rendered UI and using image processing to check it. ↩

Applying the "page object" pattern with stb-tester

Often you will hear us recommend that you should structure your test-packs as a set of library functions, plus test-cases that use those library functions. In this blog post we attempt to make this recommendation more concrete by demonstrating how to apply the “page object” pattern when writing stb-tester test-cases. Read more...

25 Aug 2015.

Handling UI elements of non-fixed size

In this blog post we use the OpenCV cv2.floodFill function to improve the robustness of test helper functions for dynamic UIs. Read more...

10 Jul 2015.

Stb-tester v23 released

Stb-tester v23 is now available, with a new option to run your testcases in random order, aiming to spend approximately the same amount of time in each testcase. Read more...

27 Jun 2015.

Stb-tester ONE v22.5 released

v22.5 for the stb-tester ONE is now available. It provides a REST API to retrieve detailed test results and artifacts, intended for integrating the stb-tester ONE with external reporting systems. Read more...

10 Jun 2015.

Stb-tester ONE v22.4 released

v22.4 for the stb-tester ONE is now available. Read more...

14 May 2015.

Stb-tester ONE v22.2 released

v22.2 for the stb-tester ONE is now available. Read more...

31 Mar 2015.

Stb-tester v22 released

Stb-tester v22 is now available. Read more...

31 Mar 2015.

A better way to organise your testcases

Up until now, stb-tester testcases were Python scripts with one testcase per file, and you referred to your testcases by the filename. Read more...

26 Mar 2015.

New documentation for stb-tester's Python API

We have overhauled the stb-tester Python API reference documentation, with more examples, introductory material, a categorised index, and a search function. Read more...

25 Sep 2014.

Upcoming stb-tester features: Re-imagining the test runner

stbt batch run works really well for stress-testing and soak-testing, but it’s lacking a few features that would be useful for functional acceptance testing: Run each test once, then re-run each failing test; run the tests in a random order or specify ordering dependencies between tests; update an external bug database with the results. Or for a continuous integration service, only run the fast tests and don’t fail the build if a test fails due to a known defect. The list goes on. Read more...

22 Sep 2014.

Upcoming stb-tester features: Composable API

Stb-tester’s wait_for_match function works well enough, but what if you want to wait for a UI element to disappear? Or when you want to check that 2 different UI elements are present in the same frame? Or when you just want to know if the element is present or not, without raising an exception? An addition to stb-tester’s Python API, due in stb-tester 0.21, will make this much easier. Read more...

23 Aug 2014.

Upcoming stb-tester features: stb-tester as a stateless service

Do you wish that your stb-tester test farm was as easy to set up as Travis CI? Want a neat way of specifying your test script’s dependencies? Or maybe you want to run stb-tester from your Windows PC? The upcoming stbt service command solves these problems. Read more...

30 Jul 2014.

Upcoming stb-tester features: Testing HTML apps and Smart TVs

Stb-tester.com Ltd is currently working with a global TV manufacturer to provide testing services for their latest development project. To support this work we have developed a lot of exciting new features in stb-tester. Read more...

10 Jun 2014.

Creating LIRC config files - part 1: theory

TL;DR version:: stb-tester reqires a LIRC config file to know how to translate key names into IR pulses. irrecord can be used to create them by recording IR signals but can have difficulty sometimes. In those cases you can provide additional information to make recording succeed which can be determined from the output of mode2 and xmode2 and by consulting the lirc config file documentation.

stb-tester 0.20 released with improved OCR

stb-tester 0.20 has been released. The key new developments are: Read more...

16 May 2014.

Stb-tester Continuous Integration with your GitHub repo

A Continuous Integration (CI) system, which runs your automated tests against every change made to your source code, is an important part of every Agile development process. With stb-tester your CI system can run integration tests on the real target device. As a demonstration we have set up stb-tester CI for a simple “Hello World” Android app at this git repo. Read more...

14 Apr 2014.

Improving OCR accuracy

TL;DR version:: stbt.ocr’s misreadings aren’t due to it not being familiar with your fonts. The root cause is that the OCR system we use (tesseract) is intended for reading the printed word. With this new understanding we’ve made improvements to OCR resulting in a 12× reduction of errors. This will be in the next stb-tester release.

Stb-tester Blog