16 May 2014. By David Röthlisberger.

A Continuous Integration (CI) system, which runs your automated tests against every change made to your source code, is an important part of every Agile development process. With stb-tester your CI system can run integration tests on the real target device. As a demonstration we have set up stb-tester CI for a simple “Hello World” Android app at this git repo.

wait_for_match(Hello World)

test_that_app_launched.py

The instant a developer pushes a change to the repository on GitHub, our CI system pulls the change, builds the app, deploys the app to a Samsung Galaxy Ace 2, launches the app on the phone, and runs a test script (shown on the right) with stb-tester to verify that the app has launched correctly and looks the way we expect it to.

The result of the test is posted back to GitHub, which displays a small green checkmark or red cross in the Pull Requests interface:

…as well as a less subtle display in the pull request’s detail view:

Clicking on the “Details” link takes you to stb-tester’s test results, which include logs and a video recording of the test run.

Automated testing, particularly integration or full-system testing, has large up-front costs. So to get the most value from test automation you want to run your tests often and early in the software development lifecycle. Read more about our approach to testing.

As an aside, stb-tester’s Android support is currently a simple proof of concept not yet merged to the stb-tester mainline (it’s on the “android” branch). An official, supported, production-ready implementation is on our roadmap – please get in touch if you are interested in commissioning this development.

Acknowledgements: Thanks to Travis CI, a unit-testing continuous integration system, for the inspiration to integrate our CI with GitHub. Travis CI is an indispensable part of stb-tester’s own software development process.