Navigating a menu

In the previous tutorial we implemented a Menu class that reads information about our device-under-test’s main menu from a frame of video. Now we will implement a Python function that will navigate to any given menu and submenu.

Using our Menu class

We can already use our Menu class in our testcases, directly. (Remember, Menu is a Frame Object that represents the currently-selected item in the main menu. It has two properties: is_visible and selection.)

For example, this is a simple test that presses up & down on the main menu 10 times, and checks that the device-under-test reacts correctly to every keypress:

from stbt import press, wait_until
from pages.menu import Menu

def test_pressing_up_and_down_on_the_main_menu():
    go_to_home()
    for _ in range(10):
        press("KEY_DOWN")
        assert wait_until(lambda: Menu().selection == "My Feed")
        press("KEY_UP")
        assert wait_until(lambda: Menu().selection == "Home")

You can watch the test running here.

Run the tests!

Once you have implemented a complete testcase, no matter how simple, run it! Ideally run it every time a developer commits a change to the device-under-test or application-under-test, in a Continuous Integration system. If that isn’t possible, at least run the tests against nightly builds. Run! Your! Tests!

Seriously, don’t write any more testcases until you are consistently running the testcases you already have. We have seen teams spend a year (no exaggeration) building elaborate test infrastructure without producing test results. Run your tests!

The testcase above seems trivial, but it can consistently reproduce what appears to be a resource leak (the Roku gets slower and slower to respond) until eventually the Roku reboots itself. If your set-top box is going to sit idle, why not leave the testcases running all night? You’ll be surprised at the bugs you’ll find.

This is especially true if you’re a set-top box manufacturer or integrator. If you’re developing an app for someone else’s device you’ll be less interested in device instabilities, but running soak tests can still find resource leaks in your own app.

Even if you don’t look at the results of every overnight soak, run the tests! That way you’ll have data if you need to answer “when did this stop working?” or “when did this start getting slower?”.

main_menu.navigate_to: The API

Usually before I implement a function, I think about how I’m going to call it from my testcases. Maybe I want to write a testcase that enters the network settings, and checks that it reports the device’s IP address:

import main_menu

def test_that_wired_network_settings_reports_ip_address():
    main_menu.navigate_to("Settings", "Network", "Wired (Ethernet)")
    # ... the rest of the testcase goes here ...

If you write the testcase code before you implement the helper functions, you’re following a process called “top-down design”, also known as “design by wishful thinking” — you pretend that you already have a function that does what you want in the most convenient way, and you worry about implementing it later. This process can make your testcases more readable.

def navigate_to(name, *submenus):
    """Navigate to the specified menu or submenu.

    Requires that you are already at the root of the main menu (for example by
    calling `go_to_home`).

    It doesn't enter the last menu, it leaves the selection on that item.

    Examples::

        # This moves the selection to "Settings":
        navigate_to("Settings")

        # This enters the Settings menu, then it enters the Network sub-menu,
        # and finally it moves the selection to "Wired (Ethernet)",
        # but it doesn't enter the "Wired (Ethernet)" sub-menu.
        navigate_to("Settings", "Network", "Wired (Ethernet)")

    """
    assert False, "Not implemented yet"  # TODO

The *submenus syntax on the first line means that the function can take a variable number of arguments. In the testcase you saw earlier, we called it like this:

main_menu.navigate_to("Settings", "Network", "Wired (Ethernet)")

In that case, name will be "Settings", and submenus will be the tuple ("Network", "Wired (Ethernet)").

Note that I have given my function a documentation string that tells other developers what the function does, and most importantly, any preconditions the caller has to satisfy before calling this function.

main_menu.navigate_to: The implementation

from stbt import press, wait_until

def navigate_to(name, *submenus):
    print "Looking for menu item '%s'" % name
    for _ in range(10):
        if wait_until(lambda: Menu().selection == name, timeout_secs=2):
            print "Found menu item '%s'" % name
            break
        press("KEY_DOWN")
    else:
        assert False, "Failed to find '%s' after pressing DOWN 10 times" % name

    if submenus:
        print "Entering menu '%s'" % name
        press("KEY_OK")
        assert wait_until(lambda: Menu().selection != name)
        navigate_to(*submenus)

Hopefully you find the implementation fairly straight-forward. For brevity I haven’t shown the docstring, which you saw earlier.

Watch it run:

The key: Separating actions from image processing

Note that navigate_to doesn’t do any image processing itself, or any other complex calculations. This separation of concerns is key to our workflow:

1. All the things that interact with the device-under-test (press, wait_until) are in navigate_to.
2. All the image processing is done in the Menu Frame Object.

In the previous tutorial we were able to test our Menu Frame Object offline — that is, without a device-under-test (except to capture some screenshots). This is a big productivity improvement, and it is also more maintainable in the long run.

The code in navigate_to interacts with the device-under-test, so the only way to test it properly is to run it against a real device. But at least it’s only 15 lines long, and pylint can test it for typos and other simple bugs.