Stb-tester news, product announcements & testing topics.
Last December our company turned 10 years old. Since founding our company 10 years ago, we have grown from a services-focused company in the first 2 years, to a primarily product-based company since 2016 with uninterrupted year-on-year sales growth and customers in 4 continents. In 2023 we had our best year yet, with sales up 50% compared to the previous year, and headcount up 25%. Read more...
Today we have released Stb-tester’s new “Smart Navigation” functions stbt.navigate_1d and stbt.navigate_grid. These functions will autonomously navigate your application’s UI, learning the structure of your menus to speed up future invocations. These are the first of many Machine Learning / AI features on Stb-tester’s roadmap; we think you’ll agree that these new APIs will greatly simplify and speed up your test development. Read more...
pyatv is an open-source tool (MIT license) for controlling Apple TV devices over the network. It uses AirPlay and other network protocols supported by the Apple TV. The most relevant features for us are: Listing the installed apps, launching an app, and sending text to the on-screen keyboard. Read more...
Stb-tester now supports the TP-Link “Kasa” brand of smart plugs. This rounds out Stb-tester’s already impressive support for a wide range of network-controlled Power Distribution Unit (PDU) models from ATEN, Aviosys, Lindy, PDUeX, and Rittal, which allows you to power-cycle the device-under-test from your test scripts. Read more...
Version 34 of Stb-tester’s “stbt” Python API is now available. This release makes important improvements to the motion-detection algorithm of stbt.wait_for_motion and stbt.press_and_wait, born from our personal experience of writing & maintaining tests for Apple TV, DirecTV, Fire TV, Sky, and Xfinity devices over the last 4 years. Read more...
In image processing, “segmentation” means finding which pixels belong to foreground objects (such as text or other GUI elements) versus background pixels. Stb-tester v33 adds a new API called segment (pronounced like the verb: segMENT) that finds the location of distinct foreground elements. Read more...
Stb-tester v33 introduces a new, easier, way of defining masks for image-processing APIs such as is_screen_black, press_and_wait, and wait_for_motion. Read more...
find_regions_by_color is a new API that finds contiguous regions of a particular color. It can be used to identify GUI elements such as the “selection” or “focus” on menus and carousels. Read more...
With the pytest support introduced in Stb-tester v33, you can use pytest-bdd to run Stb-tester from BDD specifications (also known as “Gherkin” syntax, or “Given/When/Then” acceptance criteria). Read more...
Stb-tester v33 supports running your tests with pytest, a popular Python test-runner. Pytest provides powerful functionality such as assertion introspection and fixtures. Read more...
When you push a git commit, Stb-tester uses any idle Node in your test-farm to run a “lint” check using Pylint to catch common mistakes in your code before you even run a test. Read more...
The "manual control" role allows users to control devices-under-test remotely, and see live video from the devices, without access to the test scripts in the test-pack git repository. This allows you to grant access to a third party without letting them see the test scripts you have developed which are your own intellectual property. Read more...
Stb-tester supports automated testing on the Nvidia Shield TV and Shield TV Pro. Read more...
If you started getting errors today when accessing the REST API on your Stb-tester Portal (either directly using curl or python requests, or using our stbt_rig.py command-line tool) then read on for the solution. Read more...
2021 has been a busy year for us, with major new features implemented to support specific customers’ requirements: Prometheus and Grafana dashboards for logging and reporting performance measurements and other real-time metrics; support for some Bluetooth and RF4CE remote controls via RedRat-X hardware; an interactive log viewer with frame-accurate synchronisation between each log line and the video recording of the test-run; and a new test-farm administration page. Read more...
Our VSCode IDE integration makes test-script development convenient. With it you get: Read more...
If you want to test the same app on several different platforms, you want to re-use as much test-script code as possible. For example you might have your app on Android TV, Apple TV, Fire TV, PlayStation, Roku, XBox, and possibly even some operator-specific set-top box platforms. This article shows how to use the same test scripts and the same Page Objects for all these platforms, and how to manage the cases where your app has differences in appearance or behaviour across platforms. Read more...
The Stb-tester Portal shows live video from the device-under-test, right in your web browser. Previously, this video was streamed from your Stb-tester Nodes using RTMP (Flash’s video-streaming protocol). With our latest firmware release, Stb-tester Nodes now stream this live audio & video using WebRTC – the W3C standard protocol for low-latency, real-time communication. WebRTC allowed us to achieve much better latency when you are interacting manually with your device-under-test via the Stb-tester Portal. Read more...
Stb-tester v32 added new APIs that make it much easier to navigate on-screen keyboards from your test scripts. In this tutorial we’ll model the behaviour of YouTube’s search keyboard on the Apple TV, and we’ll write a Page Object (a Python class) to navigate the keyboard from our test scripts. Read more...
Version 32 of the stb-tester core Python API is now available. We have re-architected stbt.Keyboard to easily test on-screen keyboards with multiple modes (such as lowercase, uppercase, and symbols modes). We have added a powerful new API called stbt.find_selection_from_background to easily detect the position of a “selection” or “highlight” on the screen. We have sped up stbt.ocr hugely by caching previous results for identical inputs — so now you can use OCR more freely in navigation code without worrying about slowing down your test scripts. Read more...
Stb-tester’s cloud-based portal gives you remote access to all the set-top boxes and OTT devices in your test-farm, from anywhere in the world. Many of our customers are using this capability to allow their staff, who are in quarantine during the current coronavirus pandemic, to continue QA activities from home — both manual & automated testing. Read more...
Stb-tester now provides tight integration with your organisation’s own GitHub Enterprise instance. Read more...
Stb-tester provides deep integration with Visual Studio Code, the world’s most popular IDE.1 You can now execute tests remotely (on your Stb-tester Node hardware) right from your IDE. This can greatly speed up your test-development cycle.
Pylint is a static analysis tool for Python code. This means that it analyses your code to catch common mistakes, without actually running your code. This saves a lot of development time by catching the mistakes immediately —your IDE shows red squiggles under each mistake— instead of running the test and waiting for it to reach the line of code where the mistake is. Read more...
The optional else clause after a for loop is an obscure Python feature that
is surprisingly useful when writing automated GUI tests.
Read more...
The Stb-tester Python APIs are now available on PyPI (the Python Package Index) so that you can install them on your development PC — it doesn’t matter if you’re using Windows, MacOS, or Linux. Read more...
I have written a Page Object for the “Search” page in the YouTube app on Apple TV. My Page Object has a property that reads the search term using OCR, but the OCR is mis-reading some words. Let’s investigate! Read more...
Stb-tester v31 adds support for test-scripts written in Python 3 (Python 2 is still supported, too). To migrate an existing test-pack to Python 3, we recommend the following steps. Read more...
Version 31 of the Stb-tester core Python API is available, with support for test-scripts written in Python 3 (Python 2 is still supported, too). Read more...
Stb-tester v30 includes a new feature: Searching for a reference image that has some transparent pixels. This blog post describes three ways you can use this new feature that makes writing test-scripts much easier than before. Read more...
Version 30 of the stb-tester core Python API is available. stbt.match now
supports transparency: Transparent pixels in the reference image will be
ignored when looking for a match within the video-frame. This is a huge
usability improvement when looking for buttons, selections/highlights, and
similar outlines that have dynamic content or non-rectangular shapes.
Read more...
An end-to-end test for your Alexa integration is easy to automate with Stb-tester. In this short video we issue commands to Alexa by playing pre-recorded audio files (“Alexa, play Mr. Robot”). Then we validate that the command was successful by looking at the video output from the set-top box (in this video, a Fire TV Stick). Read more...
Stb-tester now provides PyCharm integration out of the box, allowing you to run tests against the real set-top-box, right from your IDE: Read more...
The Stb-tester team will be at the IBC tradeshow in Amsterdam 14-18 September 2018. Read more...
Stb-tester has always supported Roku boxes (Roku HD, 2, 3, 4, Express, Express+, Premiere, Premiere+, and Ultra). Now, Stb-tester also supports the Roku Streaming Stick and Streaming Stick+. Read more...
Version 29 of the stb-tester core Python API is now available. This is a significant release with major new APIs for testing audio. Read more...
When you are developing new test scripts (or modifying existing ones) we recommend this workflow: Read more...
Stb-tester supports automated testing on the Amazon Fire TV and Fire TV Stick. Read more...
The Stb-tester Platform now supports Atlassian's Bamboo Server for Continuous Integration (CI). Many of our customers already use Bamboo for continuous integration, deployment, and delivery. Now, it will only take a few minutes to get your Stb-tester end-to-end tests running from your Bamboo jobs. Read more...
When you are capturing reference images for your test scripts, the Stb-tester Portal now allows you to crop the images directly in your browser. Read more...
Version 28 of the stb-tester core Python API is now available. Read more...
Stb-tester’s cloud-based portal centralizes your test results and dashboards. It allows remote testing and live monitoring from anywhere in the world — your devices can be spread across multiple network locations, and your test team can be remote too. Read more...
There are several mature open source and proprietary tools for testing the user interface of mobile apps on Android and iOS devices, but there aren’t many solutions for internet service providers or video-on-demand streaming services who want to test video playback on real mobile devices to validate their end-to-end video delivery. In this article I’ll discuss various ways of getting video frames from a mobile device for automated testing, and the advantages and trade-offs of each approach. Read more...
To test a PS4 app you need a mechanism to control the PS4 and a way to get feedback to check that your app is behaving as it should. Read more...
A Serial Terminal Server makes a serial connection available over the network, so that your application can access your devices without a direct serial connection. Typically this is used to establish a bi-directional connection between networked applications and older industrial equipment. In this article we will focus on a much simpler use case: Capturing serial logs, where we only need communication in one direction — a common use case when testing set-top boxes. Read more...
The stb-tester team will be at the IBC exhibition in Amsterdam 9-13 September 2016. Read more...
Updated 17 May 2017 with figures for our new hardware’s accuracy. Read more...
If you have already automated one or more testcases, run them! Ideally run them every time a developer commits a change to the system-under-test. If that isn’t possible, at least run the tests against nightly builds. Run! Your! Tests! Read more...
This is a technical report on our investigation into an intermittent problem with the stb-tester ONE’s infrared transmitter. Read more...
We have just published three videos showcasing the stb-tester ONE, including a detailed guide for measuring channel change times on a set-top box: Read more...
In November 2015 YouView launched on a range of Sony Bravia televisions across the UK. The biggest technical challenges of the project were in the integration, with frequent releases from YouView, Sony, MediaTek, and Google. Read more...
Stb-tester.com is pleased to announce that DTVKit will be using our stb-tester tool as part of their test automation strategy, and we will be providing our expertise and support as a Partner Member of DTVKit. Read more...
In the previous blog post we discussed the [Page Object pattern] and how it could be used to improve the readability of your test scripts. In this blog post we introduce the Frame Object pattern, a specialisation of the Page Object pattern that can be especially effective in stb-tester’s style of black-box UI testing1. Frame Objects can reduce the maintenance cost of your test-packs, particularly when the UI-under-test changes.
Often you will hear us recommend that you should structure your test-packs as a set of library functions, plus test-cases that use those library functions. In this blog post we attempt to make this recommendation more concrete by demonstrating how to apply the “page object” pattern when writing stb-tester test-cases. Read more...
stb-tester.com
irrecord can be used to create them by recording IR
signals but can have difficulty sometimes. In those cases you can provide
additional information to make recording succeed which can be determined
from the output of mode2 and xmode2 and by consulting the lirc config
file documentation.
stbt.ocr’s misreadings aren’t due to it not being familiar with your
fonts. The root cause is that the OCR system we use (tesseract) is
intended for reading the printed word. With this new understanding we’ve
made improvements to OCR resulting in a 12× reduction of errors. This will
be in the next stb-tester release.Stb-tester co-founder David Röthlisberger gave a presentation at the Google Test Automation Conference on how stb-tester was built and an overview of some of the things you can achieve with stb-tester. See the video on YouTube. Read more...