Stb-tester news, product announcements & testing topics.
07 Sep 2021.
2021 has been a busy year for us, with major new features implemented to
support specific customers’ requirements: Prometheus and Grafana dashboards
for logging and reporting performance measurements and other real-time metrics;
support for some Bluetooth and RF4CE remote controls via RedRat-X hardware;
an interactive log viewer with frame-accurate synchronisation between each
log line and the video recording of the test-run; and a new test-farm
administration page.
Read more...
11 Feb 2021.
If you want to test the same app on several different platforms, you want to
re-use as much test-script code as possible. For example you might have your
app on Android TV, Apple TV, Fire TV, PlayStation, Roku, XBox, and possibly
even some operator-specific set-top box platforms. This article shows how to
use the same test scripts and the same Page Objects for all these platforms,
and how to manage the cases where your app has differences in appearance or
behaviour across platforms.
Read more...
24 Dec 2020.
The Stb-tester Portal shows live video from the device-under-test, right in
your web browser. Previously, this video was streamed from your Stb-tester
Nodes using RTMP (Flash’s video-streaming protocol). With our latest firmware
release, Stb-tester Nodes now stream this live audio & video using WebRTC
– the W3C standard protocol for low-latency, real-time communication. WebRTC
allowed us to achieve much better latency when you are interacting manually
with your device-under-test via the Stb-tester Portal.
Read more...
02 Oct 2020.
Stb-tester v32 added new APIs that make it much easier to navigate on-screen
keyboards from your test scripts. In this tutorial we’ll model the behaviour of
YouTube’s search keyboard on the Apple TV, and we’ll write a Page Object (a
Python class) to navigate the keyboard from our test scripts.
Read more...
02 Oct 2020.
Version 32 of the stb-tester core Python API is now available. We have
re-architected stbt.Keyboard to easily test on-screen keyboards with
multiple modes (such as lowercase, uppercase, and symbols modes). We have added
a powerful new API called stbt.find_selection_from_background to easily
detect the position of a “selection” or “highlight” on the screen. We have sped
up stbt.ocr hugely by caching previous results for identical inputs — so
now you can use OCR more freely in navigation code without worrying about
slowing down your test scripts.
Read more...
30 Apr 2020.
Stb-tester’s cloud-based portal gives you remote access to all the set-top
boxes and OTT devices in your test-farm, from anywhere in the world. Many of
our customers are using this capability to allow their staff, who are in
quarantine during the current coronavirus pandemic, to continue QA activities
from home — both manual & automated testing.
Read more...
28 Jan 2020.
Stb-tester provides deep integration with Visual Studio Code, the world’s most
popular IDE.1 You can now execute tests remotely (on your Stb-tester Node
hardware) right from your IDE. This can greatly speed up your test-development
cycle.
23 Jan 2020.
Pylint is a static analysis tool for Python code. This means that it analyses
your code to catch common mistakes, without actually running your code. This
saves a lot of development time by catching the mistakes immediately —your
IDE shows red squiggles under each mistake— instead of running the test and
waiting for it to reach the line of code where the mistake is.
Read more...
05 Dec 2019.
The optional else clause after a for loop is an obscure Python feature that
is surprisingly useful when writing automated GUI tests.
Read more...
02 Dec 2019.
The Stb-tester Python APIs are now available on PyPI (the Python Package Index)
so that you can install them on your development PC — it doesn’t matter if
you’re using Windows, MacOS, or Linux.
Read more...
14 Oct 2019.
I have written a Page Object for the “Search” page in the YouTube app on
Apple TV. My Page Object has a property that reads the search term using OCR,
but the OCR is mis-reading some words. Let’s investigate!
Read more...
20 Sep 2019.
Stb-tester v31 adds support for test-scripts written in Python 3 (Python 2 is
still supported, too). To migrate an existing test-pack to Python 3, we
recommend the following steps.
Read more...
19 Sep 2019.
Version 31 of the Stb-tester core Python API is available, with support for
test-scripts written in Python 3 (Python 2 is still supported, too).
Read more...
26 Feb 2019.
Stb-tester v30 includes a new feature: Searching for a reference image that has
some transparent pixels. This blog post describes three ways you can use this
new feature that makes writing test-scripts much easier than before.
Read more...
25 Feb 2019.
Version 30 of the stb-tester core Python API is available. stbt.match now
supports transparency: Transparent pixels in the reference image will be
ignored when looking for a match within the video-frame. This is a huge
usability improvement when looking for buttons, selections/highlights, and
similar outlines that have dynamic content or non-rectangular shapes.
Read more...
22 Oct 2018.
An end-to-end test for your Alexa integration is easy to automate with
Stb-tester. In this short video we issue commands to Alexa by playing
pre-recorded audio files (“Alexa, play Mr. Robot”). Then we validate that the
command was successful by looking at the video output from the set-top box (in
this video, a Fire TV Stick).
Read more...
11 Oct 2018.
Stb-tester now provides PyCharm integration out of the box, allowing you to
run tests against the real set-top-box, right from your IDE:
Read more...
21 Aug 2018.
The Stb-tester team will be at the IBC tradeshow in Amsterdam 14-18 September
2018.
Read more...
20 Aug 2018.
Stb-tester has always supported Roku boxes (Roku HD, 2, 3, 4, Express,
Express+, Premiere, Premiere+, and Ultra). Now, Stb-tester also supports the
Roku Streaming Stick and Streaming Stick+.
Read more...
22 Jun 2018.
Version 29 of the stb-tester core Python API is now available. This is a
significant release with major new APIs for testing audio.
Read more...
10 Apr 2018.
When you are developing new test scripts (or modifying existing ones) we
recommend this workflow:
Read more...
26 Mar 2018.
Stb-tester supports automated testing on the Amazon Fire TV and Fire TV Stick.
Read more...
23 Mar 2018.
The Stb-tester Platform now supports Atlassian's Bamboo Server for Continuous
Integration (CI). Many of our customers already use Bamboo for continuous
integration, deployment, and delivery. Now, it will only take a few minutes
to get your Stb-tester end-to-end tests running from your Bamboo jobs.
Read more...
20 Mar 2018.
When you are capturing reference images for your test scripts, the Stb-tester
Portal now allows you to crop the images directly in your browser.
Read more...
06 Feb 2018.
Version 28 of the stb-tester core Python API is now available.
Read more...
17 Jan 2018.
Stb-tester’s cloud-based portal centralizes your test results and dashboards.
It allows remote testing and live monitoring from anywhere in the world —
your devices can be spread across multiple network locations, and your test
team can be remote too.
Read more...
27 Feb 2017.
Software update v27.4 for the stb-tester ONE is now available. It includes a
more convenient configuration system for large test farms, a new REST API for
searching past test results, support for controlling Android devices without a
USB connection, and more.
Read more...
21 Feb 2017.
There are several mature open source and proprietary tools for testing the user
interface of mobile apps on Android and iOS devices, but there aren’t many
solutions for internet service providers or video-on-demand streaming services
who want to test video playback on real mobile devices to validate their
end-to-end video delivery. In this article I’ll discuss various ways of getting
video frames from a mobile device for automated testing, and the advantages and
trade-offs of each approach.
Read more...
23 Jan 2017.
Software update v27.3 for the stb-tester ONE is now available. It adds support
for the Sony PlayStation 3 & 4, Android devices, and more!
Read more...
05 Jan 2017.
To test a PS4 app you need a mechanism to control the PS4 and a way to get
feedback to check that your app is behaving as it should.
Read more...
12 Dec 2016.
The stb-tester camera allows you to test the user interface of devices that
don’t have an HDMI output, such as Smart TVs and mobile devices.
Read more...
28 Oct 2016.
A Serial Terminal Server makes a serial connection available over the network,
so that your application can access your devices without a direct serial
connection. Typically this is used to establish a bi-directional connection
between networked applications and older industrial equipment. In this article
we will focus on a much simpler use case: Capturing serial logs, where we only
need communication in one direction — a common use case when testing set-top
boxes.
Read more...
04 Oct 2016.
Software update v26.6 for the stb-tester ONE is now available. It contains a
JPEG thumbnail alternative to the Flash video in the web interface, and a
bugfix for inode exhaustion on the stb-tester ONE’s test-results partition.
Read more...
20 Sep 2016.
Selenium is a test automation tool for web applications. It tests your
web app by looking at the DOM – you can check that your web page contains a
particular element like a div or a button, identified by id or css selector or
xpath expression; once the element is located, you can check that it contains a
particular text or other elements. However you can’t really check what your web
app looks like. If you have complex CSS it can be all too easy to introduce
unintended defects in your app’s visual appearance.
Read more...
08 Aug 2016.
The stb-tester team will be at the IBC exhibition in Amsterdam 9-13 September
2016.
Read more...
22 Jul 2016.
Software update v26.2 for the stb-tester ONE is now available, with new
features for managing & monitoring large test farms.
Read more...
13 Jul 2016.
Stb-tester v26 is now available, with support for frame-accurate performance
measurements.
Read more...
05 Jul 2016.
Updated 17 May 2017 with figures for our new hardware’s accuracy.
Read more...
29 Jun 2016.
If you have already automated one or more testcases, run them! Ideally run them
every time a developer commits a change to the system-under-test. If that isn’t
possible, at least run the tests against nightly builds. Run! Your! Tests!
Read more...
21 Jun 2016.
We have just published an extensive tutorial about our recommended workflow for
implementing automated test scripts, using stbt.FrameObject to extract
information from the screen, using stbt lint and stbt auto-selftest to
test your scripts locally, and using stbt-docker to develop on Mac or
Windows.
Read more...
03 Jun 2016.
Stb-tester v25 is now available, with features that significantly improve the
maintenance cost of your test scripts:
Read more...
27 May 2016.
Software update v24.8 for the stb-tester ONE is now available. It fixes a
regression introduced in v24.7 that caused the stb-tester ONE’s old USB
infrared transmitter to fail for signals longer than 125 milliseconds.
Read more...
26 May 2016.
Software update v24.7 for the stb-tester ONE is now available.
Read more...
26 May 2016.
This is a technical report on our investigation into an intermittent problem
with the stb-tester ONE’s infrared transmitter.
Read more...
06 May 2016.
How to create a custom remote control image for the stb-tester ONE’s “manual
control” web interface:
Read more...
22 Apr 2016.
Software update v24.5 for the stb-tester ONE is now available. This release
delivers a minor UI improvement, and a bugfix that affects a small number of
users.
Read more...
29 Feb 2016.
We have just published three videos showcasing the stb-tester ONE, including a
detailed guide for measuring channel change times on a set-top box:
Read more...
11 Feb 2016.
Software update v24.4 for the stb-tester ONE is now available, with the ability
to run tests from any git branch, and several bugfixes.
Read more...
20 Jan 2016.
Software update v24.3 for the stb-tester ONE is now available. This includes
better feedback when you have errors in your infrared configuration file, and
other small usability improvements & bugfixes.
Read more...
14 Jan 2016.
Stb-tester v24 is now available, with many small Python API improvements and
fixes.
Read more...
11 Jan 2016.
Software update v23.7 for the stb-tester ONE is now available. This includes a new test-results interface with much better search capabilities, plus some minor fixes & updates.
Read more...
18 Dec 2015.
In November 2015 YouView launched on a range of Sony Bravia televisions across
the UK. The biggest technical challenges of the project were in the
integration, with frequent releases from YouView, Sony, MediaTek, and Google.
Read more...
16 Dec 2015.
Software update v23.6 for the stb-tester ONE is now available. This is a bugfix release.
Read more...
02 Dec 2015.
Stb-tester.com is pleased to announce that DTVKit will be using our
stb-tester tool as part of their test automation strategy, and we will
be providing our expertise and support as a Partner Member of DTVKit.
Read more...
09 Nov 2015.
Software update v23.5 for the stb-tester ONE is now available. It has a
technology preview of upcoming improvements to the results viewer, as well as
bugfixes.
Read more...
24 Sep 2015.
In the previous blog post we discussed the frame object pattern and how it
could lead to greater agility and lower maintenance costs thanks to improved
self-testability. This blog post discusses a technique to get the maximum
value from this testability at near-zero cost.
Read more...
08 Sep 2015.
In the previous blog post we discussed the [Page Object pattern] and how it
could be used to improve the readability of your test scripts. In this blog
post we introduce the Frame Object pattern, a specialisation of the Page Object
pattern that can be especially effective in stb-tester’s style of black-box UI
testing1. Frame Objects can reduce the maintenance cost of your test-packs,
particularly when the UI-under-test changes.
02 Sep 2015.
Often you will hear us recommend that you should structure your test-packs as a
set of library functions, plus test-cases that use those library functions. In
this blog post we attempt to make this recommendation more concrete by
demonstrating how to apply the “page object” pattern when writing
stb-tester test-cases.
Read more...
25 Aug 2015.
In this blog post we use the OpenCV [cv2.floodFill][floodfill]
function to improve the robustness of test helper functions for dynamic UIs.
Read more...
10 Jul 2015.
Stb-tester v23 is now available, with a new option to run your testcases in
random order, aiming to spend approximately the same amount of time in each
testcase.
Read more...
27 Jun 2015.
v22.5 for the stb-tester ONE is now available. It provides a REST API to
retrieve detailed test results and artifacts, intended for integrating
the stb-tester ONE with external reporting systems.
Read more...
10 Jun 2015.
v22.4 for the stb-tester ONE is now available.
Read more...
14 May 2015.
v22.2 for the stb-tester ONE is now available.
Read more...
31 Mar 2015.
Stb-tester v22 is now available.
Read more...
31 Mar 2015.
Up until now, stb-tester testcases were Python scripts with one testcase per
file, and you referred to your testcases by the filename.
Read more...
26 Mar 2015.
We have overhauled the stb-tester Python API reference documentation, with more
examples, introductory material, a categorised index, and a search function.
Read more...
25 Sep 2014.
stbt batch run works really well for stress-testing and soak-testing, but
it’s lacking a few features that would be useful for functional acceptance
testing: Run each test once, then re-run each failing test; run the tests in a
random order or specify ordering dependencies between tests; update an external
bug database with the results. Or for a continuous integration service, only
run the fast tests and don’t fail the build if a test fails due to a known
defect. The list goes on.
Read more...
22 Sep 2014.
Stb-tester’s wait_for_match function works well enough, but what if you want
to wait for a UI element to disappear? Or when you want to check that 2
different UI elements are present in the same frame? Or when you just want to
know if the element is present or not, without raising an exception? An
addition to stb-tester’s Python API, due in stb-tester 0.21, will make this
much easier.
Read more...
23 Aug 2014.
Do you wish that your stb-tester test farm was as easy to set up as
Travis CI? Want a neat way of specifying your test script’s dependencies? Or
maybe you want to run stb-tester from your Windows PC? The upcoming stbt
service command solves these problems.
Read more...
30 Jul 2014.
Stb-tester.com Ltd is currently working with a global TV manufacturer to provide
testing services for their latest development project. To support this work we
have developed a lot of exciting new features in stb-tester.
Read more...
10 Jun 2014.
- TL;DR version:
- stb-tester reqires a LIRC config file to know how to translate key names
into IR pulses.
irrecord can be used to create them by recording IR
signals but can have difficulty sometimes. In those cases you can provide
additional information to make recording succeed which can be determined
from the output of mode2 and xmode2 and by consulting the lirc config
file documentation.
Read more...
10 Jun 2014.
stb-tester 0.20 has been released. The key new developments are:
Read more...
16 May 2014.
A Continuous Integration (CI) system, which runs your automated tests against
every change made to your source code, is an important part of every Agile
development process. With stb-tester your CI system can run integration tests
on the real target device. As a demonstration we have set up stb-tester CI
for a simple “Hello World” Android app at this git repo.
Read more...
14 Apr 2014.
- TL;DR version:
stbt.ocr’s misreadings aren’t due to it not being familiar with your
fonts. The root cause is that the OCR system we use (tesseract) is
intended for reading the printed word. With this new understanding we’ve
made improvements to OCR resulting in a 12× reduction of errors. This will
be in the next stb-tester release.
Read more...
09 Apr 2014.
stb-tester 0.19.1 has been released. The only change since 0.19 is a fix to
the Fedora packages to make them installable on Fedora 19. See the [release
notes] for (not much) more information.
Read more...
04 Apr 2014.
stb-tester 0.19 has been released. This is a beta release intended to allow
for wider testing rather than production use.
Read more...
28 Jan 2014.
stb-tester 0.18 has been released with support for OCR and enhancements to the test runner - now called stbt batch.
Read more...
17 Dec 2013.
stb-tester 0.17 has been released. Among other improvementes image matching is now 2-6x faster.
Read more...
12 Dec 2013.
stb-tester maintainer and original author David Röthlisberger gave a presentation on stb-tester at the GStreamer conference in Edinburgh in October. It includes:
Read more...
01 May 2013.
stb-tester maintainer and original author David Röthlisberger gave a presentation on how stb-tester was built and an overview of some of the things you can achieve with stb-tester. See the video on YouTube.
Read more...
stb-tester.com