01 May 2020.

Tags: Test Development

In part 2 we saw how to test an on-screen keyboard by modelling the keyboard as a Directed Graph. We specified the model by writing out each button and each transition explicitly; in this article we will see more convenient ways of specifying the model.

_kb = stbt.Keyboard("""
    a b KEY_RIGHT
    a g KEY_DOWN
    b a KEY_LEFT
    b h KEY_DOWN
    g a KEY_UP
    g h KEY_RIGHT
    h b KEY_UP
    h g KEY_LEFT
Detail of YouTube's on-screen keyboard, with the corresponding model definition from part 2

The Python code in the figure above (“a b KEY_RIGHT”, etc.) is the “edge list” that we defined in part 2. It specifies which key we need to press (on the remote control) to navigate between states in the on-screen keyboard.

In this app, the keyboard is a perfect grid so we can generate the edge list programatically. In part 1 we already modelled this grid layout using stbt.Grid, for the purpose of mapping the white selection to the corresponding letter:

KEYBOARD_GRID = stbt.Grid(
    region=stbt.Region(x=125, y=165, right=430, bottom=470),
BOTTOM_GRID = stbt.Grid(
    region=stbt.Region(x=125, y=470, right=430, bottom=525),
    data=[[" ", "DELETE", "CLEAR"]])
KEYBOARD_GRID in red and BOTTOM_GRID in yellow, defined in part 1

Stb-tester provides an API to convert a stbt.Grid into a directed graph suitable for stbt.Keyboard: it’s called stbt.grid_to_navigation_graph. For example, the following code defines the navigation paths within KEYBOARD_GRID (shown in red in the screenshot above):

_kb = stbt.Keyboard(

But now we’re missing the navigation between space, backspace, and clear in the second grid (which has wider columns than the top grid). We can combine two directed graphs into a single one, like this:

_graph = networkx.compose_all([
_kb = stbt.Keyboard(_graph)

networkx is an open-source Python library that Stb-tester uses for representing directed graphs. stbt.grid_to_navigation_graph returns a networkx.DiGraph, and networkx.compose_all allows merging two or more graphs into a single one.

Now we have defined the navigation within each of the two grids, but we’re still missing the navigation going down from the bottom of the first grid into the second grid, and vice-versa:

Transitions between KEYBOARD_GRID and BOTTOM_GRID

These aren’t entirely grid-like so we need to specify them explicitly, and then convert them to a networkx DiGraph using stbt.Keyboard.parse_edgelist, like this:

_graph = networkx.compose_all([
        5 SPACE KEY_DOWN
        6 SPACE KEY_DOWN
        9 CLEAR KEY_DOWN
        0 CLEAR KEY_DOWN
        SPACE 5 KEY_UP
        SPACE 6 KEY_UP
        CLEAR 9 KEY_UP
        CLEAR 0 KEY_UP
_kb = stbt.Keyboard(_graph)

As you can see, we still needed to specify some transitions explicitly — but only 12 of them, not 136 (see the diff compared to part 2). Less code means that we are less likely to have made a mistake in our model.

See the full code from this tutorial here.

Addendum: Adding nodes and edges to a graph using the networkx API

The networkx DiGraph (created by stbt.grid_to_navigation_graph or stbt.Keyboard.parse_edgelist) can be extended by calling its add_edge method. For example:

_graph = networkx.compose_all([
for top, bottom in [("5", "SPACE"),
                    ("6", "SPACE"),
                    ("7", "BACKSPACE"),
                    ("8", "BACKSPACE"),
                    ("9", "CLEAR"),
                    ("0", "CLEAR")]:
    _graph.add_edge(top, bottom, key="KEY_DOWN")
    _graph.add_edge(bottom, top, key="KEY_UP")
_kb = stbt.Keyboard(_graph)

Internally, stbt.Keyboard uses a networkx.DiGraph where each node in the graph is a string (the character corresponding to that key on the keyboard), and each edge between 2 nodes has a data attribute called key that is the key on the remote control that you need to press to move from the source node to the target node.

Warning: In future versions of the Stb-tester Python API, the format of stbt.Keyboard’s internal graph representation may change. So if you manipulate the graph directly with add_edge (like the example above) be prepared to change your code when upgrading your test-pack to new versions of the Stb-tester API. The description in the previous paragraph is true as of Stb-tester version 31. Public Stb-tester APIs (stbt.grid_to_navigation_graph and stbt.Keyboard.parse_edgelist) are stable, so if you only use those APIs you don’t need to worry about this warning.