Navigating a menu
In the previous tutorial we implemented a
class that reads information about our device-under-test’s main menu from a
frame of video. Now we will implement a Python function that will navigate to
any given menu and submenu.
Using our Selection class
We can already use our
Selection class in our testcases, directly. (Remember,
Selection is a Frame Object that represents the currently-selected item in the main menu. It has two properties:
For example, this is a simple test that presses up & down on the main menu 10 times, and checks that the device-under-test reacts correctly to every keypress:
You can watch the test running here.
Run the tests!
Once you have implemented a complete testcase, no matter how simple, run it! Ideally run it every time a developer commits a change to the device-under-test or application-under-test, in a Continuous Integration system. If that isn’t possible, at least run the tests against nightly builds. Run! Your! Tests!
Seriously, don’t write any more testcases until you are consistently running the testcases you already have. We have seen teams spend a year (no exaggeration) building elaborate test infrastructure without producing test results. Run your tests!
The testcase above seems trivial, but it can consistently reproduce what appears to be a resource leak (the Roku gets slower and slower to respond) until eventually the Roku reboots itself. If your set-top box is going to sit idle, why not leave the testcases running all night? You’ll be surprised at the bugs you’ll find.
This is especially true if you’re a set-top box manufacturer or integrator. If you’re developing an app for someone else’s device you’ll be less interested in device instabilities, but running soak tests can still find resource leaks in your own app.
Even if you don’t look at the results of every overnight soak, run the tests! That way you’ll have data if you need to answer “when did this stop working?” or “when did this start getting slower?”.
main_menu.navigate_to: The API
Usually before I implement a function, I think about how I’m going to call it from my testcases. Maybe I want to write a testcase that enters the network settings, and checks that it reports the device’s IP address:
If you write the testcase code before you implement the helper functions, you’re following a process called “top-down design”, also known as “design by wishful thinking” — you pretend that you already have a function that does what you want in the most convenient way, and you worry about implementing it later. This process can make your testcases more readable.
*submenus syntax on the first line means that the function can take a
variable number of arguments. In the testcase you saw earlier, we called it
In that case,
name will be
submenus will be the tuple
("Network", "Wired (Ethernet)").
Note that I have given my function a documentation string that tells other developers what the function does, and most importantly, any preconditions the caller has to satisfy before calling this function.
main_menu.navigate_to: The implementation
Hopefully you find the implementation fairly straight-forward. For brevity I haven’t shown the docstring, which you saw earlier.
Watch it run:
The key: Separating actions from image processing
navigate_to doesn’t do any image processing itself, or any other
complex calculations. This separation of concerns is key to our workflow:
- All the things that interact with the device-under-test (
wait_until) are in
- All the image processing is done in the
In the previous tutorial we were able to test our
Selection Frame Object offline — that is, without a device-under-test
(except to capture some screenshots). This is a big productivity improvement,
and it is also more maintainable in the long run.
The code in
navigate_to interacts with the device-under-test, so the only way
to test it properly is to run it against a real device. But at least it’s only
15 lines long, and you can still use stbt lint to test it for typos and other