stbt Python API¶

Remote control¶

• stbt.press: Send the specified key-press to the system-under-test.
• stbt.pressing: Press and hold a key until some other condition is met.
• stbt.press_and_wait: Press a key, then wait for the screen to change, then wait for it to stop changing.
• stbt.press_until_match: Call press as many times as necessary to find the specified image.

Network-based protocols¶

Some devices (such as the Roku and some Smart TVs) can be controlled via HTTP or other network protocols. You can use any Python networking library to make network requests to such devices (to install third-party Python libraries see Customising the test-run environment). We recommend the Python requests library, which is already installed.

Alexa and Google Home¶

• stbt.audio.play_audio_file: Play an audio clip (for example “Alexa, play Teletubbies”) to test integration of your device with voice-controlled devices like Alexa or Google Home.

Searching for an image¶

• stbt.wait_for_match: Search for the specified image in the video stream; raise stbt.MatchTimeout if not found within a certain timeout.
• stbt.match: Search for the specified image in a single video frame; return a truthy/falsey stbt.MatchResult.
• stbt.match_all: Search for all instances of the specified image in a single video frame.

Use stbt.match with assert and stbt.wait_until for a more flexible alternative to stbt.wait_for_match. For example, to wait for an image to disappear:

stbt.press("KEY_CLOSE")
assert wait_until(lambda: not stbt.match("guide.png"))

Searching for text using OCR (optical character recognition)¶

• stbt.match_text: Search for the specified text in a single video frame; return a truthy/falsey stbt.TextMatchResult.
• stbt.ocr: Read the text present in a video frame.

Searching for motion¶

• stbt.wait_for_motion: Search for motion in the video stream; raise stbt.MotionTimeout if no motion is found within a certain timeout.
• stbt.detect_motion: Generator that yields a sequence of one stbt.MotionResult for each frame processed from the system-under-test, indicating whether any motion was detected.

Miscellaneous video APIs¶

• stbt.is_screen_black: Check whether the video frame is completely black.

Audio APIs¶

Audio input:

• stbt.audio.get_rms_volume: Calculate the average RMS volume over a given duration.
• stbt.audio.wait_for_volume_change: Wait for changes in the RMS audio volume. Can detect the start of content playback or unmuting; bleeps or clicks while navigating the UI; or beeps in an A/V sync video.
• stbt.audio.audio_chunks: Low-level API to get raw audio samples for custom analysis.

Audio output:

• stbt.audio.play_audio_file: Play an audio file through the Stb-tester Node’s “audio out” jack. Useful for testing integration of your device with Alexa or Google Home.

stbt.as_precondition¶

stbt.as_precondition(message)¶

Context manager that replaces test failures with test errors.

Stb-tester’s reports show test failures (that is, UITestFailure or AssertionError exceptions) as red results, and test errors (that is, unhandled exceptions of any other type) as yellow results. Note that wait_for_match, wait_for_motion, and similar functions raise a UITestFailure when they detect a failure. By running such functions inside an as_precondition context, any UITestFailure or AssertionError exceptions they raise will be caught, and a PreconditionError will be raised instead.

When running a single testcase hundreds or thousands of times to reproduce an intermittent defect, it is helpful to mark unrelated failures as test errors (yellow) rather than test failures (red), so that you can focus on diagnosing the failures that are most likely to be the particular defect you are looking for. For more details see Test failures vs. errors.

Parameters:	message (str) – A description of the precondition. Word this positively: “Channels tuned”, not “Failed to tune channels”.
Raises:	PreconditionError if the wrapped code block raises a UITestFailure or AssertionError.

Example:

def test_that_the_on_screen_id_is_shown_after_booting():
    channel = 100

    with stbt.as_precondition("Tuned to channel %s" % channel):
        mainmenu.close_any_open_menu()
        channels.goto_channel(channel)
        power.cold_reboot()
        assert channels.is_on_channel(channel)

    stbt.wait_for_match("on-screen-id.png")

stbt.audio.audio_chunks¶

stbt.audio.audio_chunks(time_index=None)¶

Low-level API to get raw audio samples.

audio_chunks returns an iterator of AudioChunk objects. Each one contains 100ms to 5s of mono audio samples (see AudioChunk for the data format).

audio_chunks keeps a buffer of 10s of audio samples. time_index allows the caller to access these old samples. If you read from the returned iterator too slowly you may miss some samples. The returned iterator will skip these old samples and silently re-sync you at -10s. You can detect this situation by comparing the .end_time of the previous chunk to the .time of the current one.

Parameters:	time_index (int or float) – Time from which audio samples should be yielded. This is an epoch time compatible with time.time(). Defaults to the current time as given by time.time().
Returns:	An iterator yielding AudioChunk objects
Return type:	Iterator[AudioChunk]

Added in v29. audio_chunks is a premium API only available to stb-tester.com customers.

stbt.audio.AudioChunk¶

class stbt.audio.AudioChunk¶

A sequence of audio samples.

An AudioChunk object is what you get from audio_chunks. It is a subclass of numpy.ndarray. An AudioChunk is a 1-D array containing audio samples in 32-bit floating point format (numpy.float32) between -1.0 and 1.0.

In addition to the members inherited from numpy.ndarray, AudioChunk defines the following attributes:

Variables:	time (float) – The wall-clock time of the first audio sample in this chunk, as number of seconds since the unix epoch (1970-01-01T00:00:00Z). This is the same format used by the Python standard library function time.time. rate (int) – Number of samples per second. This will typically be 48000. duration (float) – The duration of this audio chunk in seconds. end_time (float) – time + duration.

AudioChunk supports slicing using Python’s [x:y] syntax, so the above attributes will be updated appropriately on the returned slice.

Added in v29. AudioChunk is a premium API only available to stb-tester.com customers.

stbt.audio.get_rms_volume¶

stbt.audio.get_rms_volume(duration_secs=3.0, stream=None)¶

Calculate the average RMS volume of the audio over the given duration.

For example, to check that your mute button works:

stbt.press('KEY_MUTE')
time.sleep(1)  # <- give it some time to take effect
assert get_rms_volume().amplitude < 0.001  # -60 dB

Parameters:	duration_secs (int or float) – The window over which you should average, in seconds. Defaults to 3s in accordance with short-term loudness from the EBU TECH 3341 specification. stream (Iterator[AudioChunk]) – Audio stream to measure. Defaults to audio_chunks().
Raises:	ZeroDivisionError – If duration_secs is shorter than one sample or stream contains no samples.
Returns:	An object with the following attributes: amplitude (float) – The RMS amplitude over the specified window. This is a value between 0.0 (absolute silence) and 1.0 (a full-range square wave). time (float) – The start of the window, as number of seconds since the unix epoch (1970-01-01T00:00Z). This is compatible with time.time() and stbt.Frame.time. duration_secs (float) – The window size, in seconds. Typically this will be the same as passed into get_rms_volume().

Added in v29. get_rms_volume is a premium API only available to stb-tester.com customers.

stbt.audio.play_audio_file¶

stbt.audio.play_audio_file(filename)¶

Play an audio file through the Stb-tester Node’s “audio out” jack.

Useful for testing integration of your device with Alexa or Google Home.

Parameters:	filename (str) – The audio file to play (for example a WAV or MP3 file committed to your test-pack). Filenames should be relative paths. This uses the same path lookup algorithm as stbt.load_image.

Added in v29.

stbt.audio.VolumeChangeDirection¶

class stbt.audio.VolumeChangeDirection¶

An enumeration.

LOUDER = 1¶

QUIETER = -1¶

stbt.audio.VolumeChangeTimeout¶

exception stbt.audio.VolumeChangeTimeout¶

Bases: AssertionError

stbt.audio.wait_for_volume_change¶

stbt.audio.wait_for_volume_change(direction=VolumeChangeDirection.LOUDER, stream=None, window_size_secs=0.4, threshold_db=10.0, noise_floor_amplitude=0.0003, timeout_secs=10)¶

Wait for changes in the RMS audio volume.

This can be used to listen for the start of content, or for bleeps and bloops when navigating the UI. It returns after the first significant volume change. This function tries hard to give accurate timestamps for when the volume changed. It works best for sudden changes like a beep.

This function detects changes in volume using a rolling window. The RMS volume is calculated over a rolling window of size window_size_secs. For every sample this function compares the RMS volume in the window preceeding the sample, to the RMS volume in the window following the sample. The ratio of the two volumes determines whether the volume change is significant or not.

Example: Measure the latency of the mute button:

start = time.time()
stbt.press('KEY_MUTE')
quiet = wait_for_volume_change(
    direction=VolumeChangeDirection.QUIETER,
    stream=audio_chunks(time_index=start))
print "MUTE latency: %0.3f s" % (quiet.time - start_time)

Example: Measure A/V sync between “beep.png” being displayed and a beep being heard:

video = wait_for_match("beep.png")
audio = wait_for_volume_change(
    stream=audio_chunks(time_index=video.time - 0.5),
    window_size_secs=0.01)
print "a/v sync: %i ms" % (video.time - audio.time) * 1000

Parameters:	direction (VolumeChangeDirection) – Whether we should wait for the volume to increase or decrease. Defaults to VolumeChangeDirection.LOUDER. stream (Iterator returned by audio_chunks) – Audio stream to listen to. Defaults to audio_chunks(). Postcondition: the stream will be positioned at the time of the volume change. window_size_secs (int) – The time over which the RMS volume should be averaged. Defaults to 0.4 (400ms) in accordance with momentary loudness from the EBU TECH 3341 specification. Decrease this if you want to detect bleeps shorter than 400ms duration. threshold_db (float) – This controls sensitivity to volume changes. A volume change is considered significant if the ratio between the volume before and the volume afterwards is greater than threshold_db. With threshold_db=10 (the default) and direction=VolumeChangeDirection.LOUDER the RMS volume must increase by 10 dB (a factor of 3.16 in amplitude). With direction=VolumeChangeDirection.QUIETER the RMS volume must fall by 10 dB. noise_floor_amplitude (float) – This is used to avoid ZeroDivisionError exceptions. The change from an amplitude of 0 to 0.1 is ∞ dB. This isn’t very practical to deal with so we consider 0 amplitude to be this non-zero noise_floor_amplitude instead. It defaults to ~0.0003 (-70dBov). Increase this value if there is some sort of background noise that you want to ignore. timeout_secs (float) – Timeout in seconds. If no significant volume change is found within this time, VolumeChangeTimeout will be raised and your test will fail.
Raises:	VolumeChangeTimeout – If no volume change is detected before timeout_secs.
Returns:	An object with the following attributes: direction (VolumeChangeDirection) – This will be either VolumeChangeDirection.LOUDER or VolumeChangeDirection.QUIETER as given to wait_for_volume_change. rms_before (see get_rms_volume) – The RMS volume averaged over the window immediately before the volume change. Use result.rms_before.amplitude to get the RMS amplitude as a float. rms_after (see get_rms_volume) – The RMS volume averaged over the window immediately after the volume change. difference_db (float) – Ratio between rms_after and rms_before, in decibels. difference_amplitude (float) – Absolute difference between the rms_after and rms_before. This is a number in the range -1.0 to +1.0. time (float) – The time of the volume change, as number of seconds since the unix epoch (1970-01-01T00:00:00Z). This is the same format used by the Python standard library function time.time() and stbt.Frame.time. window_size_secs (float) – The size of the window over which the volume was averaged, in seconds.

Added in v29. wait_for_volume_change is a premium API only available to stb-tester.com customers.

stbt.ConfigurationError¶

exception stbt.ConfigurationError¶

Bases: Exception

An error with your stbt configuration file.

stbt.ConfirmMethod¶

class stbt.ConfirmMethod¶

An enumeration.

NONE = 'none'¶

ABSDIFF = 'absdiff'¶

NORMED_ABSDIFF = 'normed-absdiff'¶

stbt.crop¶

stbt.crop(frame, region)¶

Returns an image containing the specified region of frame.

Parameters:	frame (stbt.Frame or numpy.ndarray) – An image in OpenCV format (for example as returned by frames, get_frame and load_image, or the frame parameter of MatchResult).
Returns:	An OpenCV image (numpy.ndarray) containing the specified region of the source frame. This is a view onto the original data, so if you want to modify the cropped image call its copy() method first.

Added in v28.

stbt.detect_motion¶

stbt.detect_motion(timeout_secs=10, noise_threshold=None, mask=None, region=Region.ALL, frames=None)¶

Generator that yields a sequence of one MotionResult for each frame processed from the device-under-test’s video stream.

The MotionResult indicates whether any motion was detected.

Use it in a for loop like this:

for motionresult in stbt.detect_motion():
    ...

In most cases you should use wait_for_motion instead.

Parameters:

timeout_secs (int or float or None) – A timeout in seconds. After this timeout the iterator will be exhausted. Thas is, a for loop like for m in detect_motion(timeout_secs=10) will terminate after 10 seconds. If timeout_secs is None then the iterator will yield frames forever. Note that you can stop iterating (for example with break) at any time. noise_threshold (float) – The amount of noise to ignore. This is only useful with noisy analogue video sources. Valid values range from 0 (all differences are considered noise; a value of 0 will never report motion) to 1.0 (any difference is considered motion). This defaults to 0.84. You can override the global default value by setting noise_threshold in the [motion] section of .stbt.conf. mask (str or numpy.ndarray) – A black & white image that specifies which part of the image to search for motion. White pixels select the area to analyse; black pixels select the area to ignore. The mask must be the same size as the video frame. This can be a string (a filename that will be resolved as per load_image) or a single-channel image in OpenCV format. region (Region) – Only analyze the specified region of the video frame. If you specify both region and mask, the mask must be the same size as the region. frames (Iterator[stbt.Frame]) – An iterable of video-frames to analyse. Defaults to stbt.frames().

Added in v28: The region parameter.

Added in v29: The frames parameter.

stbt.draw_text¶

stbt.draw_text(text, duration_secs=3)¶

Write the specified text to the output video.

Parameters:	text (str) – The text to write. duration_secs (int or float) – The number of seconds to display the text.

stbt.Frame¶

class stbt.Frame¶

A frame of video.

A Frame is what you get from stbt.get_frame and stbt.frames. It is a subclass of numpy.ndarray, which is the type that OpenCV uses to represent images. Data is stored in 8-bit, 3 channel BGR format.

In addition to the members inherited from numpy.ndarray, Frame defines the following attributes:

Variables:	time (float) – The wall-clock time when this video-frame was captured, as number of seconds since the unix epoch (1970-01-01T00:00:00Z). This is the same format used by the Python standard library function time.time.

stbt.FrameObject¶

class stbt.FrameObject(frame=None)¶

Base class for user-defined Page Objects.

FrameObjects are Stb-tester’s implementation of the Page Object pattern. A FrameObject is a class that uses Stb-tester APIs like stbt.match() and stbt.ocr() to extract information from the screen, and it provides a higher-level API in the vocabulary and user-facing concepts of your own application.

Based on Martin Fowler’s PageObject diagram

Stb-tester uses a separate instance of your FrameObject class for each frame of video captured from the device-under-test (hence the name “Frame Object”). Stb-tester provides additional tooling for writing, testing, and maintenance of FrameObjects.

To define your own FrameObject class:

• Derive from stbt.FrameObject.
• Define an is_visible property (using Python’s @property decorator) that returns True or False.
• Define any other properties for information that you want to extract from the frame.
• Inside each property, when you call an image-processing function (like stbt.match or stbt.ocr) you must specify the parameter frame=self._frame.

The following behaviours are provided automatically by the FrameObject base class:

• Truthiness: A FrameObject instance is considered “truthy” if it is visible. Any other properties (apart from is_visible) will return None if the object isn’t visible.
• Immutability: FrameObjects are immutable, because they represent information about a specific frame of video – in other words, an instance of a FrameOject represents the state of the device-under-test at a specific point in time. If you define any methods that change the state of the device-under-test, they should return a new FrameObject instance instead of modifying self.
• Caching: Each property will be cached the first time is is used. This allows writing testcases in a natural way, while expensive operations like ocr will only be done once per frame.

The FrameObject base class defines several convenient methods and attributes (see below).

Further reading:

• The PageObject pattern by Martin Fowler.
• Tutorial: Using FrameObjects to extract information from the screen.
• Stb-tester’s Object Repository GUI for creating, debugging, and viewing FrameObjects.

Added in v30: _fields and refresh.

_fields¶

A tuple containing the names of the public properties.

__init__(frame=None)¶

The default constructor takes an optional frame of video; if the frame is not provided, it will grab a frame from the device-under-test.

If you override the constructor in your derived class (for example to accept additional parameters), make sure to accept an optional frame parameter and supply it to the super-class’s constructor.

__repr__()¶

The object’s string representation includes all its public properties.

__bool__()¶

Delegates to is_visible. The object will only be considered True if it is visible.

__eq__(other)¶

Two instances of the same FrameObject type are considered equal if the values of all the public properties match, even if the underlying frame is different. All falsey FrameObjects of the same type are equal.

__hash__()¶

Two instances of the same FrameObject type are considered equal if the values of all the public properties match, even if the underlying frame is different. All falsey FrameObjects of the same type are equal.

refresh(frame=None, **kwargs)¶

Returns a new FrameObject instance with a new frame. self is not modified.

refresh is used by navigation functions that modify the state of the device-under-test.

By default refresh returns a new object of the same class as self, but you can override the return type by implementing refresh in your derived class.

Any additional keyword arguments are passed on to __init__.

stbt.frames¶

stbt.frames(timeout_secs=None)¶

Generator that yields video frames captured from the device-under-test.

Parameters:	timeout_secs (int or float or None) – A timeout in seconds. After this timeout the iterator will be exhausted. That is, a for loop like for f in frames(timeout_secs=10) will terminate after 10 seconds. If timeout_secs is None (the default) then the iterator will yield frames forever. Note that you can stop iterating (for example with break) at any time.
Return type:	Iterator[stbt.Frame]
Returns:	An iterator of frames in OpenCV format (stbt.Frame).

Changed in v29: Returns Iterator[stbt.Frame] instead of Iterator[(stbt.Frame, int)]. Use the Frame’s time attribute instead.

stbt.get_config¶

stbt.get_config(section, key, default=None, type_=str)¶

Read the value of key from section of the test-pack configuration file.

For example, if your configuration file looks like this:

[test_pack]
stbt_version = 30

[my_company_name]
backend_ip = 192.168.1.23

then you can read the value from your test script like this:

backend_ip = stbt.get_config("my_company_name", "backend_ip")

This searches in the .stbt.conf file at the root of your test-pack, and in the config/test-farm/<hostname>.conf file matching the hostname of the stb-tester device where the script is running. Values in the host-specific config file override values in .stbt.conf. See Configuration files for more details.

Raises ConfigurationError if the specified section or key is not found, unless default is specified (in which case default is returned).

stbt.get_frame¶

stbt.get_frame()¶

Grabs a video frame captured from the device-under-test.

Returns:	The latest video frame in OpenCV format (a stbt.Frame).

stbt.Grid¶

class stbt.Grid(region, cols=None, rows=None, data=None)¶

A grid with items arranged left to right, then down.

For example a keyboard, or a grid of posters, arranged like this:

ABCDE
FGHIJ
KLMNO

All items must be the same size, and the spacing between them must be consistent.

This class is useful for converting between pixel coordinates on a screen, to x & y indexes into the grid positions.

Parameters:

region (Region) – Where the grid is on the screen. cols (int) – Width of the grid, in number of columns. rows (int) – Height of the grid, in number of rows. data – A 2D array (list of lists) containing data to associate with each cell. The data can be of any type. For example, if you are modelling a grid-shaped keyboard, the data could be the letter at each grid position. If data is specified, then cols and rows are optional.

class Cell¶

A single cell in a Grid.

Don’t construct Cells directly; create a Grid instead.

Variables:

index (int) – The cell’s 1D index into the grid, starting from 0 at the top left, counting along the top row left to right, then the next row left to right, etc. position (Position) – The cell’s 2D index (x, y) into the grid (zero-based). For example in this grid “I” is index 8 and position (x=3, y=1): ABCDE FGHIJ KLMNO region (Region) – Pixel coordinates (relative to the entire frame) of the cell’s bounding box. data – The data corresponding to the cell, if data was specified when you created the Grid.

get(index=None, position=None, region=None, data=None)¶

Retrieve a single cell in the Grid.

For example, let’s say that you’re looking for the selected item in a grid by matching a reference image of the selection border. Then you can find the (x, y) position in the grid of the selection, like this:

selection = stbt.match("selection.png")
cell = grid.get(region=selection.region)
position = cell.position

You must specify one (and only one) of index, position, region, or data. For the meaning of these parameters see Grid.Cell. A negative index counts backwards from the end of the grid (so -1 is the bottom right position).

Returns:	The Grid.Cell that matches the specified query; raises IndexError if the index/position/region is out of bounds or the data is not found.

stbt.grid_to_navigation_graph¶

stbt.grid_to_navigation_graph(grid)¶

Generate a Graph that describes navigation between cells in the grid.

Creates a networkx.DiGraph (Directed Graph) that models cells in the grid as nodes in the graph. Each edge in the graph has a key attribute set to “KEY_LEFT”, “KEY_RIGHT”, “KEY_UP”, or “KEY_DOWN”, corresponding to the keypress that will move a selection from one node to another.

Parameters:	grid (Grid) – The grid to model. If the Grid has data associated with the cells, each node in the graph will be named with the corresponding cell’s data; otherwise the nodes are numbered according to the cell’s index. This means that the cell’s data must be hashable so that it can be used as a networkx node.
Returns:	A networkx.DiGraph.

For example, to create a graph suitable as the graph parameter of stbt.Keyboard:

grid = stbt.Grid(region, data=["ABCDEFG",
                               "HIJKLMN",
                               "OPQRSTU",
                               "VWXYZ-'"])
keyboard = stbt.Keyboard(grid_to_navigation_graph(grid))

stbt.is_screen_black¶

stbt.is_screen_black(frame=None, mask=None, threshold=None, region=Region.ALL)¶

Check for the presence of a black screen in a video frame.

Parameters:

frame (stbt.Frame or numpy.ndarray) – If this is specified it is used as the video frame to check; otherwise a new frame is grabbed from the device-under-test. This is an image in OpenCV format (for example as returned by frames and get_frame). mask (str or numpy.ndarray) – A black & white image that specifies which part of the image to analyse. White pixels select the area to analyse; black pixels select the area to ignore. The mask must be the same size as the video frame. This can be a string (a filename that will be resolved as per load_image) or a single-channel image in OpenCV format. threshold (int) – Even when a video frame appears to be black, the intensity of its pixels is not always 0. To differentiate almost-black from non-black pixels, a binary threshold is applied to the frame. The threshold value is in the range 0 (black) to 255 (white). The global default can be changed by setting threshold in the [is_screen_black] section of .stbt.conf. region (Region) – Only analyze the specified region of the video frame. If you specify both region and mask, the mask must be the same size as the region.

Returns:

An object that will evaluate to true if the frame was black, or false if not black. The object has the following attributes: black (bool) – True if the frame was black. frame (stbt.Frame) – The video frame that was analysed.

Added in v28: The region parameter.

Added in v29: Return an object with a frame attribute, instead of bool.

stbt.Keyboard¶

class stbt.Keyboard(graph, mask=None, navigate_timeout=20)¶

Helper for navigating an on-screen keyboard using the remote control.

You customize for the appearance & behaviour of the keyboard you’re testing by specifying two things:

• A Page Object that can tell you which key is currently selected on the screen. See the page parameter to enter_text and navigate_to, below.
• A Directed Graph that specifies the navigation between every key on the keyboard (for example on a qwerty keyboard: when Q is selected, pressing KEY_RIGHT on the remote control goes to W, and so on). See the graph parameter below.

The constructor takes the following parameters:

Parameters:

graph (str or networkx.DiGraph) – A specification of the complete navigation graph (state machine) between adjacent keys, as a multiline string where each line is in the format <start_node> <end_node> <action>. For example, the specification for a qwerty keyboard might look like this: ''' Q W KEY_RIGHT Q A KEY_DOWN W Q KEY_LEFT <etc> ''' For nodes that enter a character, use that character as the node name. For the space-bar use SPACE. For other nodes that don’t enter a character when pressed, use a descriptive name such as CLEAR or ENTER (these nodes won’t be used by enter_text but you can use them as a target of navigate_to). On some keyboards, buttons like the space-bar are wider than other buttons and the navigation away from the button depends on the previous state. Our graph can’t model this state, so specify all the possible transitions from the button. For example, on a qwerty keyboard: SPACE C KEY_UP SPACE V KEY_UP SPACE B KEY_UP SPACE N KEY_UP SPACE M KEY_UP For advanced users: instead of a string, graph can be a networkx.DiGraph where each edge has an attribute called key with a value like "KEY_RIGHT". If your keyboard’s buttons are positioned in a regular grid, you can use stbt.grid_to_navigation_graph to generate this graph (or part of the graph, and then you can add any irregular connections explicitly with networkx.DiGraph.add_edge). See also Keyboard.parse_edgelist. mask (str) – A mask to use when calling stbt.press_and_wait to determine when the current selection has finished moving. If the search page has a blinking cursor you need to mask out the region where the cursor can appear, as well as any other regions with dynamic content (such as a picture-in-picture with live TV). See stbt.press_and_wait for more details about the mask. navigate_timeout (int or float) – Timeout (in seconds) for navigate_to. In practice navigate_to should only time out if you have a bug in your graph state machine specification.

class Selection¶

Type that your Page Object’s selection property can return.

Has two attributes:

• text (str) — The selected letter or button.
• region (stbt.Region) — The position on screen of the selection / highlight.

Is falsey if text and region are both None.

enter_text(text, page, verify_every_keypress=False)¶

Enter the specified text using the on-screen keyboard.

Parameters:

text (str) – The text to enter. If your keyboard only supports a single case then you need to convert the text to uppercase or lowercase, as appropriate, before passing it to this method. page (stbt.FrameObject) – An instance of a stbt.FrameObject sub-class that describes the appearance of the on-screen keyboard. It must implement the following: selection (str or Keyboard.Selection) — property that returns the name of the currently selected character (for example “A” or ” ”) or button (for example “CLEAR” or “SEARCH”). This property can return a string, or an object with a text attribute that is a string. For grid-shaped keyboards, you can implement this property using stbt.Grid to map from the region of the selection (highlight) to the corresponding letter; see the example below. The page instance that you provide must represent the current state of the device-under-test. verify_every_keypress (bool) – If True, we will read the selected key after every keypress and assert that it matches the model (graph). If False (the default) we will only verify the selected key corresponding to each of the characters in text. For example: to get from Q to D on a qwerty keyboard you need to press KEY_DOWN, KEY_RIGHT, KEY_RIGHT. The default behaviour will only verify that the selected key is D after pressing KEY_RIGHT the last time. This is faster, and closer to the way a human uses the on-screen keyboard. Set this to True to help debug your model if enter_text is behaving incorrectly.

Typically your FrameObject will provide its own enter_text method, so your test scripts won’t call this Keyboard class directly. For example:

class YouTubeSearch(stbt.FrameObject):
    _kb = stbt.Keyboard('''
        A B KEY_RIGHT
        ...etc...
        ''')
    letters = stbt.Grid(region=...,
                        data=["ABCDEFG",
                              "HIJKLMN",
                              "OPQRSTU",
                              "VWXYZ-'"])
    space_row = stbt.Grid(region=...,
                          data=[[" ", "CLEAR", "SEARCH"]])

    @property
    def is_visible(self):
        ...  # implementation not shown

    @property
    def selection(self):
        m = stbt.match("keyboard-selection.png", frame=self._frame)
        if not m:
            return stbt.Keyboard.Selection(None, None)
        try:
            text = self.letters.get(region=m.region).data
        except IndexError:
            text = self.space_row.get(region=m.region).data
        return stbt.Keyboard.Selection(text, m.region)

    def enter_text(self, text):
        page = self
        page = self._kb.enter_text(text.upper(), page)
        self._kb.navigate_to("SEARCH", page)
        stbt.press("KEY_OK")

    def navigate_to(self, target):
        return self._kb.navigate_to(target, page=self)

navigate_to(target, page, verify_every_keypress=False)¶

Move the selection to the specified character.

Note that this won’t press KEY_OK on the target, it only moves the selection there.

Parameters:	target (str) – The key or button to navigate to, for example “A”, ” ”, or “CLEAR”. page (stbt.FrameObject) – See enter_text. verify_every_keypress (bool) – See enter_text.
Returns:	A new FrameObject instance of the same type as page, reflecting the device-under-test’s new state after the navigation completed.

static parse_edgelist(graph)¶

Create a networkx.DiGraph from a string specification of the graph.

This is useful when you want to specify part of the keyboard’s navigation graph programmatically using stbt.grid_to_navigation_graph (for the parts of the keyboard that are laid out in a grid and behave regularly) but you still need to specify some extra edges that behave differently. For example:

letters = stbt.Grid(...)
space_bar = stbt.Keyboard.parse_edgelist('''
    C SPACE KEY_DOWN
    V SPACE KEY_DOWN
    B SPACE KEY_DOWN
    SPACE C KEY_UP
    SPACE V KEY_UP
    SPACE B KEY_UP
''')
keyboard = stbt.Keyboard(networkx.compose_all([
    stbt.grid_to_navigation_graph(letters),
    space_bar]))

Parameters:	graph (str) – See the Keyboard constructor.
Returns:	A new networkx.DiGraph instance.

stbt.load_image¶

stbt.load_image(filename, flags=None)¶

Find & read an image from disk.

If given a relative filename, this will search in the directory of the Python file that called load_image, then in the directory of that file’s caller, etc. This allows you to use load_image in a helper function, and then call that helper function from a different Python file passing in a filename relative to the caller.

Finally this will search in the current working directory. This allows loading an image that you had previously saved to disk during the same test run.

This is the same lookup algorithm used by stbt.match and similar functions.

Parameters:	filename (str or unicode) – A relative or absolute filename. flags – Flags to pass to cv2.imread.
Returns:	An image in OpenCV format — that is, a numpy.ndarray of 8-bit values. With the default flags parameter this will be 3 channels BGR, or 4 channels BGRA if the file has transparent pixels.
Raises:	IOError if the specified path doesn’t exist or isn’t a valid image file.

• Added in v28.
• Changed in v30: Include alpha (transparency) channel if the file has transparent pixels.

stbt.match¶

stbt.match(image, frame=None, match_parameters=None, region=Region.ALL)¶

Search for an image in a single video frame.

Parameters:

image (string or numpy.ndarray) – The image to search for. It can be the filename of a png file on disk, or a numpy array containing the pixel data in 8-bit BGR format. If the image has an alpha channel, any transparent pixels are ignored. Filenames should be relative paths. See stbt.load_image for the path lookup algorithm. 8-bit BGR numpy arrays are the same format that OpenCV uses for images. This allows generating reference images on the fly (possibly using OpenCV) or searching for images captured from the device-under-test earlier in the test script. frame (stbt.Frame or numpy.ndarray) – If this is specified it is used as the video frame to search in; otherwise a new frame is grabbed from the device-under-test. This is an image in OpenCV format (for example as returned by frames and get_frame). match_parameters (MatchParameters) – Customise the image matching algorithm. See MatchParameters for details. region (Region) – Only search within the specified region of the video frame.

Returns:

A MatchResult, which will evaluate to true if a match was found, false otherwise.

Added in v30: Support transparency in the reference image, and new match method MatchMethod.SQDIFF.

stbt.match_all¶

stbt.match_all(image, frame=None, match_parameters=None, region=Region.ALL)¶

Search for all instances of an image in a single video frame.

Arguments are the same as match.

Returns:	An iterator of zero or more MatchResult objects (one for each position in the frame where image matches).

Examples:

all_buttons = list(stbt.match_all("button.png"))

for match_result in stbt.match_all("button.png"):
    # do something with match_result here
    ...

stbt.match_text¶

stbt.match_text(text, frame=None, region=Region.ALL, mode=OcrMode.PAGE_SEGMENTATION_WITHOUT_OSD, lang=None, tesseract_config=None, case_sensitive=False, upsample=True, text_color=None, text_color_threshold=None, engine=None, char_whitelist=None)¶

Search for the specified text in a single video frame.

This can be used as an alternative to match, searching for text instead of an image.

Parameters:	text (unicode) – The text to search for. frame – See ocr. region – See ocr. mode – See ocr. lang – See ocr. tesseract_config – See ocr. upsample – See ocr. text_color – See ocr. text_color_threshold – See ocr. engine – See ocr. char_whitelist – See ocr. case_sensitive (bool) – Ignore case if False (the default).
Returns:	A TextMatchResult, which will evaluate to True if the text was found, false otherwise.

For example, to select a button in a vertical menu by name (in this case “TV Guide”):

m = stbt.match_text("TV Guide")
assert m.match
while not stbt.match('selected-button.png').region.contains(m.region):
    stbt.press('KEY_DOWN')

Added in v28: The upsample and text_color parameters.

Added in v29: The text_color_threshold parameter.

Added in v30: The engine parameter and support for Tesseract v4.

Added in v31: The char_whitelist parameter.

stbt.MatchMethod¶

class stbt.MatchMethod¶

An enumeration.

SQDIFF = 'sqdiff'¶

SQDIFF_NORMED = 'sqdiff-normed'¶

CCORR_NORMED = 'ccorr-normed'¶

CCOEFF_NORMED = 'ccoeff-normed'¶

stbt.MatchParameters¶

class stbt.MatchParameters(match_method=None, match_threshold=None, confirm_method=None, confirm_threshold=None, erode_passes=None)¶

Parameters to customise the image processing algorithm used by match, wait_for_match, and press_until_match.

You can change the default values for these parameters by setting a key (with the same name as the corresponding python parameter) in the [match] section of .stbt.conf. But we strongly recommend that you don’t change the default values from what is documented here.

You should only need to change these parameters when you’re trying to match a reference image that isn’t actually a perfect match – for example if there’s a translucent background with live TV visible behind it; or if you have a reference image of a button’s background and you want it to match even if the text on the button doesn’t match.

Parameters:

match_method (MatchMethod) – The method to be used by the first pass of stb-tester’s image matching algorithm, to find the most likely location of the reference image within the larger source image. For details see OpenCV’s cv2.matchTemplate. Defaults to MatchMethod.SQDIFF. match_threshold (float) – Overall similarity threshold for the image to be considered a match. This threshold applies to the average similarity across all pixels in the image. Valid values range from 0 (anything is considered to match) to 1 (the match has to be pixel perfect). Defaults to 0.98. confirm_method (ConfirmMethod) – The method to be used by the second pass of stb-tester’s image matching algorithm, to confirm that the region identified by the first pass is a good match. The first pass often gives false positives: It can report a “match” for an image with obvious differences, if the differences are local to a small part of the image. The second pass is more CPU-intensive, but it only checks the position of the image that the first pass identified. The allowed values are: ConfirmMethod.NONE:   Do not confirm the match. This is useful if you know that the reference image is different in some of the pixels. For example to find a button, even if the text inside the button is different. ConfirmMethod.ABSDIFF:   Compare the absolute difference of each pixel from the reference image against its counterpart from the candidate region in the source video frame. ConfirmMethod.NORMED_ABSDIFF:   Normalise the pixel values from both the reference image and the candidate region in the source video frame, then compare the absolute difference as with ABSDIFF. This method is better at noticing differences in low-contrast images (compared to the ABSDIFF method), but it isn’t suitable for reference images that don’t have any structure (that is, images that are a single solid color without any lines or variation). This is the default method, with a default confirm_threshold of 0.70. confirm_threshold (float) – The minimum allowed similarity between any given pixel in the reference image and the corresponding pixel in the source video frame, as a fraction of the pixel’s total luminance range. Unlike match_threshold, this threshold applies to each pixel individually: Any pixel that exceeds this threshold will cause the match to fail (but see erode_passes below). Valid values range from 0 (less strict) to 1.0 (more strict). Useful values tend to be around 0.84 for ABSDIFF, and 0.70 for NORMED_ABSDIFF. Defaults to 0.70. erode_passes (int) – After the ABSDIFF or NORMED_ABSDIFF absolute difference is taken, stb-tester runs an erosion algorithm that removes single-pixel differences to account for noise and slight rendering differences. Useful values are 1 (the default) and 0 (to disable this step).

stbt.MatchResult¶

class stbt.MatchResult¶

The result from match.

Variables:

time (float) – The time at which the video-frame was captured, in seconds since 1970-01-01T00:00Z. This timestamp can be compared with system time (time.time()). match (bool) – True if a match was found. This is the same as evaluating MatchResult as a bool. That is, if result: will behave the same as if result.match:. region (Region) – Coordinates where the image was found (or of the nearest match, if no match was found). first_pass_result (float) – Value between 0 (poor) and 1.0 (excellent match) from the first pass of stb-tester’s image matching algorithm (see MatchParameters for details). frame (Frame) – The video frame that was searched, as given to match. image – The reference image that was searched for, as given to match.

stbt.MatchTimeout¶

exception stbt.MatchTimeout¶

Bases: _stbt.types.UITestFailure

Exception raised by wait_for_match.

Variables:	screenshot (Frame) – The last video frame that wait_for_match checked before timing out. expected (str) – Filename of the image that was being searched for. timeout_secs (int or float) – Number of seconds that the image was searched for.

stbt.MotionResult¶

class stbt.MotionResult¶

The result from detect_motion and wait_for_motion.

Variables:

time (float) – The time at which the video-frame was captured, in seconds since 1970-01-01T00:00Z. This timestamp can be compared with system time (time.time()). motion (bool) – True if motion was found. This is the same as evaluating MotionResult as a bool. That is, if result: will behave the same as if result.motion:. region (Region) – Bounding box where the motion was found, or None if no motion was found. frame (Frame) – The video frame in which motion was (or wasn’t) found.

Added in v28: The frame attribute.

stbt.MotionTimeout¶

exception stbt.MotionTimeout¶

Bases: _stbt.types.UITestFailure

Exception raised by wait_for_motion.

Variables:	screenshot (Frame) – The last video frame that wait_for_motion checked before timing out. mask (str or None) – Filename of the mask that was used, if any. timeout_secs (int or float) – Number of seconds that motion was searched for.

stbt.ocr¶

stbt.ocr(frame=None, region=Region.ALL, mode=OcrMode.PAGE_SEGMENTATION_WITHOUT_OSD, lang=None, tesseract_config=None, tesseract_user_words=None, tesseract_user_patterns=None, upsample=True, text_color=None, text_color_threshold=None, engine=None, char_whitelist=None)¶

Return the text present in the video frame as a Unicode string.

Perform OCR (Optical Character Recognition) using the “Tesseract” open-source OCR engine.

Parameters:

frame – If this is specified it is used as the video frame to process; otherwise a new frame is grabbed from the device-under-test. This is an image in OpenCV format (for example as returned by frames and get_frame). region (Region) – Only search within the specified region of the video frame. mode (OcrMode) – Tesseract’s layout analysis mode. lang (str) – The three-letter ISO-639-3 language code of the language you are attempting to read; for example “eng” for English or “deu” for German. More than one language can be specified by joining with ‘+’; for example “eng+deu” means that the text to be read may be in a mixture of English and German. This defaults to “eng” (English). You can override the global default value by setting lang in the [ocr] section of .stbt.conf. You may need to install the tesseract language pack; see installation instructions here. tesseract_config (dict) – Allows passing configuration down to the underlying OCR engine. See the tesseract documentation for details. tesseract_user_words (unicode string, or list of unicode strings) – List of words to be added to the tesseract dictionary. To replace the tesseract system dictionary altogether, also set tesseract_config={'load_system_dawg': False, 'load_freq_dawg': False}. tesseract_user_patterns (unicode string, or list of unicode strings) – List of patterns to add to the tesseract dictionary. The tesseract pattern language corresponds roughly to the following regular expressions: tesseract regex ========= =========== \c [a-zA-Z] \d [0-9] \n [a-zA-Z0-9] \p [:punct:] \a [a-z] \A [A-Z] \* * upsample (bool) – Upsample the image 3x before passing it to tesseract. This helps to preserve information in the text’s anti-aliasing that would otherwise be lost when tesseract binarises the image. This defaults to True; you should only disable it if you are doing your own pre-processing on the image. text_color (3-element tuple of integers between 0 and 255, BGR order) – Color of the text. Specifying this can improve OCR results when tesseract’s default thresholding algorithm doesn’t detect the text, for example white text on a light-colored background or text on a translucent overlay. text_color_threshold (int) – The threshold to use with text_color, between 0 and 255. Defaults to 25. You can override the global default value by setting text_color_threshold in the [ocr] section of .stbt.conf. engine (OcrEngine) – The OCR engine to use. Defaults to OcrEngine.TESSERACT. You can override the global default value by setting engine in the [ocr] section of .stbt.conf. char_whitelist (unicode string) – String of characters that are allowed. Useful when you know that the text is only going to contain numbers or IP addresses, for example so that tesseract won’t think that a zero is the letter o. Note that Tesseract 4.0’s LSTM engine ignores char_whitelist.

Added in v28: The upsample and text_color parameters.

Added in v29: The text_color_threshold parameter.

Added in v30: The engine parameter and support for Tesseract v4.

Added in v31: The char_whitelist parameter.

stbt.OcrEngine¶

class stbt.OcrEngine¶

An enumeration.

TESSERACT = 0¶

Tesseract’s “legacy” OCR engine (v3). Recommended.

LSTM = 1¶

Tesseract v4’s “Long Short-Term Memory” neural network. Not recommended for reading menus, buttons, prices, numbers, times, etc, because it hallucinates text that isn’t there when the input isn’t long prose.

TESSERACT_AND_LSTM = 2¶

Combine results from Tesseract legacy & LSTM engines. Not recommended because it favours the result from the LSTM engine too heavily.

DEFAULT = 3¶

Default engine, based on what is installed.

stbt.OcrMode¶

class stbt.OcrMode¶

Options to control layout analysis and assume a certain form of image.

For a (brief) description of each option, see the tesseract(1) man page.

ORIENTATION_AND_SCRIPT_DETECTION_ONLY = 0¶

PAGE_SEGMENTATION_WITH_OSD = 1¶

PAGE_SEGMENTATION_WITHOUT_OSD_OR_OCR = 2¶

PAGE_SEGMENTATION_WITHOUT_OSD = 3¶

SINGLE_COLUMN_OF_TEXT_OF_VARIABLE_SIZES = 4¶

SINGLE_UNIFORM_BLOCK_OF_VERTICALLY_ALIGNED_TEXT = 5¶

SINGLE_UNIFORM_BLOCK_OF_TEXT = 6¶

SINGLE_LINE = 7¶

SINGLE_WORD = 8¶

SINGLE_WORD_IN_A_CIRCLE = 9¶

SINGLE_CHARACTER = 10¶

SPARSE_TEXT = 11¶

SPARSE_TEXT_WITH_OSD = 12¶

RAW_LINE = 13¶

stbt.Position¶

class stbt.Position¶

A point with x and y coordinates.

stbt.PreconditionError¶

exception stbt.PreconditionError¶

Exception raised by as_precondition.

stbt.press¶

stbt.press(key, interpress_delay_secs=None, hold_secs=None)¶

Send the specified key-press to the device under test.

Parameters:

key (str) – The name of the key/button. If you are using infrared control, this is a key name from your lircd.conf configuration file. If you are using HDMI CEC control, see the available key names here. Note that some devices might not understand all of the CEC commands in that list. interpress_delay_secs (int or float) – The minimum time to wait after a previous key-press, in order to accommodate the responsiveness of the device-under-test. This defaults to 0.3. You can override the global default value by setting interpress_delay_secs in the [press] section of .stbt.conf. hold_secs (int or float) – Hold the key down for the specified duration (in seconds). Currently this is implemented for the infrared, HDMI CEC, and Roku controls. There is a maximum limit of 60 seconds.

Returns:

An object with the following attributes: key (str) – the name of the key that was pressed. start_time (float) – the time just before the keypress started (in seconds since the unix epoch, like time.time() and stbt.Frame.time). end_time (float) – the time when transmission of the keypress signal completed. frame_before (stbt.Frame) – the most recent video-frame just before the keypress started. Typically this is used by functions like stbt.press_and_wait to detect when the device-under-test reacted to the keypress.

• Added in v29: The hold_secs parameter.
• Added in v30: Returns an object with keypress timings, instead of None.

stbt.press_and_wait¶

stbt.press_and_wait(key, region=stbt.Region.ALL, mask=None, timeout_secs=10, stable_secs=1)¶

Press a key, then wait for the screen to change, then wait for it to stop changing.

This can be used to wait for a menu selection to finish moving before attempting to OCR at the selection’s new position; or to measure the duration of animations; or to measure how long it takes for a screen (such as an EPG) to finish populating.

Parameters:

key (str) – The name of the key to press (passed to stbt.press). region (stbt.Region) – Only look at the specified region of the video frame. mask (str) – The filename of a black & white image that specifies which part of the video frame to look at. White pixels select the area to analyse; black pixels select the area to ignore. You can’t specify region and mask at the same time. timeout_secs (int or float) – A timeout in seconds. This function will return a falsey value if the transition didn’t complete within this number of seconds from the key-press. stable_secs – A duration in seconds. The screen must stay unchanged (within the specified region or mask) for this long, for the transition to be considered “complete”.

Returns:

An object that will evaluate to true if the transition completed, false otherwise. It has the following attributes: key (str) – The name of the key that was pressed. frame (stbt.Frame) – If successful, the first video frame when the transition completed; if timed out, the last frame seen. status (TransitionStatus) – Either START_TIMEOUT, STABLE_TIMEOUT, or COMPLETE. If it’s COMPLETE, the whole object will evaluate as true. press_time (float) – When the key-press completed. animation_start_time (float) – When animation started after the key-press (or None if timed out). end_time (float) – When animation completed (or None if timed out). duration (float) – Time from press_time to end_time (or None if timed out). animation_duration (float) – Time from animation_start_time to end_time (or None if timed out). All times are measured in seconds since 1970-01-01T00:00Z; the timestamps can be compared with system time (the output of time.time()).

stbt.pressing¶

stbt.pressing(key, interpress_delay_secs=None)¶

Context manager that will press and hold the specified key for the duration of the with code block.

For example, this will hold KEY_RIGHT until wait_for_match finds a match or times out:

with stbt.pressing("KEY_RIGHT"):
    stbt.wait_for_match("last-page.png")

The same limitations apply as stbt.press’s hold_secs parameter.

This function was added in v29.

stbt.press_until_match¶

stbt.press_until_match(key, image, interval_secs=None, max_presses=None, match_parameters=None, region=Region.ALL)¶

Call press as many times as necessary to find the specified image.

Parameters:	key – See press. image – See match. interval_secs (int or float) – The number of seconds to wait for a match before pressing again. Defaults to 3. You can override the global default value by setting interval_secs in the [press_until_match] section of .stbt.conf. max_presses (int) – The number of times to try pressing the key and looking for the image before giving up and raising MatchTimeout. Defaults to 10. You can override the global default value by setting max_presses in the [press_until_match] section of .stbt.conf. match_parameters – See match. region – See match.
Returns:	MatchResult when the image is found.
Raises:	MatchTimeout if no match is found after timeout_secs seconds.

Added in v28: The region parameter.

stbt.Region¶

class stbt.Region¶

Region(x, y, width=width, height=height) or Region(x, y, right=right, bottom=bottom)

Rectangular region within the video frame.

For example, given the following regions a, b, and c:

- 01234567890123
0 ░░░░░░░░
1 ░a░░░░░░
2 ░░░░░░░░
3 ░░░░░░░░
4 ░░░░▓▓▓▓░░▓c▓
5 ░░░░▓▓▓▓░░▓▓▓
6 ░░░░▓▓▓▓░░░░░
7 ░░░░▓▓▓▓░░░░░
8     ░░░░░░b░░
9     ░░░░░░░░░

>>> a = Region(0, 0, width=8, height=8)
>>> b = Region(4, 4, right=13, bottom=10)
>>> c = Region(10, 4, width=3, height=2)
>>> a.right
8
>>> b.bottom
10
>>> b.contains(c), a.contains(b), c.contains(b)
(True, False, False)
>>> b.extend(x=6, bottom=-4) == c
True
>>> a.extend(right=5).contains(c)
True
>>> a.width, a.extend(x=3).width, a.extend(right=-3).width
(8, 5, 5)
>>> c.replace(bottom=10)
Region(x=10, y=4, right=13, bottom=10)
>>> Region.intersect(a, b)
Region(x=4, y=4, right=8, bottom=8)
>>> Region.intersect(a, b) == Region.intersect(b, a)
True
>>> Region.intersect(c, b) == c
True
>>> print(Region.intersect(a, c))
None
>>> print(Region.intersect(None, a))
None
>>> Region.intersect(a)
Region(x=0, y=0, right=8, bottom=8)
>>> Region.intersect()
Region.ALL
>>> quadrant = Region(x=float("-inf"), y=float("-inf"), right=0, bottom=0)
>>> quadrant.translate(2, 2)
Region(x=-inf, y=-inf, right=2, bottom=2)
>>> c.translate(x=-9, y=-3)
Region(x=1, y=1, right=4, bottom=3)
>>> Region.intersect(Region.ALL, c) == c
True
>>> Region.ALL
Region.ALL
>>> print(Region.ALL)
Region.ALL
>>> c.above()
Region(x=10, y=-inf, right=13, bottom=4)
>>> c.below()
Region(x=10, y=6, right=13, bottom=inf)
>>> a.right_of()
Region(x=8, y=0, right=inf, bottom=8)
>>> a.right_of(width=2)
Region(x=8, y=0, right=10, bottom=8)
>>> c.left_of()
Region(x=-inf, y=4, right=10, bottom=6)

x¶

The x coordinate of the left edge of the region, measured in pixels from the left of the video frame (inclusive).

y¶

The y coordinate of the top edge of the region, measured in pixels from the top of the video frame (inclusive).

right¶

The x coordinate of the right edge of the region, measured in pixels from the left of the video frame (exclusive).

bottom¶

The y coordinate of the bottom edge of the region, measured in pixels from the top of the video frame (exclusive).

width¶

The width of the region, measured in pixels.

height¶

The height of the region, measured in pixels.

x, y, right, bottom, width and height can be infinite — that is, float("inf") or -float("inf").

static from_extents()¶

Create a Region using right and bottom extents rather than width and height.

Typically you’d use the right and bottom parameters of the Region constructor instead, but this factory function is useful if you need to create a Region from a tuple.

>>> extents = (4, 4, 13, 10)
>>> Region.from_extents(*extents)
Region(x=4, y=4, right=13, bottom=10)

static bounding_box(*args)¶

Returns:	The smallest region that contains all the given regions.

>>> a = Region(50, 20, right=60, bottom=40)
>>> b = Region(20, 30, right=30, bottom=50)
>>> c = Region(55, 25, right=70, bottom=35)
>>> Region.bounding_box(a, b)
Region(x=20, y=20, right=60, bottom=50)
>>> Region.bounding_box(b, b)
Region(x=20, y=30, right=30, bottom=50)
>>> Region.bounding_box(None, b)
Region(x=20, y=30, right=30, bottom=50)
>>> Region.bounding_box(b, None)
Region(x=20, y=30, right=30, bottom=50)
>>> Region.bounding_box(b, Region.ALL)
Region.ALL
>>> print(Region.bounding_box(None, None))
None
>>> print(Region.bounding_box())
None
>>> Region.bounding_box(b)
Region(x=20, y=30, right=30, bottom=50)
>>> Region.bounding_box(a, b, c) == \
...     Region.bounding_box(a, Region.bounding_box(b, c))
True

Changed in v30: bounding_box can take an arbitrary number of region arguments, rather than exactly two.

static intersect(*args)¶

Returns:	The intersection of the passed regions, or None if the regions don’t intersect.

Any parameter can be None (an empty Region) so intersect is commutative and associative.

Changed in v30: intersect can take an arbitrary number of region arguments, rather than exactly two.

to_slice()¶

A 2-dimensional slice suitable for indexing a stbt.Frame.

contains(other)¶

Returns:	True if other is entirely contained within self.

translate(x=0, y=0)¶

Returns:	A new region with the position of the region adjusted by the given amounts.

extend(x=0, y=0, right=0, bottom=0)¶

Returns:	A new region with the edges of the region adjusted by the given amounts.

replace(x=None, y=None, width=None, height=None, right=None, bottom=None)¶

Returns:	A new region with the edges of the region set to the given coordinates.

This is similar to extend, but it takes absolute coordinates within the image instead of adjusting by a relative number of pixels.

dilate(n)¶

Expand the region by n px in all directions.

>>> Region(20, 30, right=30, bottom=50).dilate(3)
Region(x=17, y=27, right=33, bottom=53)

erode(n)¶

Shrink the region by n px in all directions.

>>> Region(20, 30, right=30, bottom=50).erode(3)
Region(x=23, y=33, right=27, bottom=47)
>>> print(Region(20, 30, 10, 20).erode(5))
None

above(height=inf)¶

Returns:	A new region above the current region, extending to the top of the frame (or to the specified height).

below(height=inf)¶

Returns:	A new region below the current region, extending to the bottom of the frame (or to the specified height).

right_of(width=inf)¶

Returns:	A new region to the right of the current region, extending to the right edge of the frame (or to the specified width).

left_of(width=inf)¶

Returns:	A new region to the left of the current region, extending to the left edge of the frame (or to the specified width).

stbt.TextMatchResult¶

class stbt.TextMatchResult¶

The result from match_text.

Variables:

time (float) – The time at which the video-frame was captured, in seconds since 1970-01-01T00:00Z. This timestamp can be compared with system time (time.time()). match (bool) – True if a match was found. This is the same as evaluating MatchResult as a bool. That is, if result: will behave the same as if result.match:. region (Region) – Bounding box where the text was found, or None if the text wasn’t found. frame (Frame) – The video frame that was searched, as given to match_text. text (unicode) – The text that was searched for, as given to match_text.

stbt.UITestFailure¶

exception stbt.UITestFailure¶

Bases: Exception

The test failed because the device under test didn’t behave as expected.

Inherit from this if you need to define your own test-failure exceptions.

stbt.wait_for_match¶

stbt.wait_for_match(image, timeout_secs=10, consecutive_matches=1, match_parameters=None, region=Region.ALL, frames=None)¶

Search for an image in the device-under-test’s video stream.

Parameters:	image – The image to search for. See match. timeout_secs (int or float or None) – A timeout in seconds. This function will raise MatchTimeout if no match is found within this time. consecutive_matches (int) – Forces this function to wait for several consecutive frames with a match found at the same x,y position. Increase consecutive_matches to avoid false positives due to noise, or to wait for a moving selection to stop moving. match_parameters – See match. region – See match. frames (Iterator[stbt.Frame]) – An iterable of video-frames to analyse. Defaults to stbt.frames().
Returns:	MatchResult when the image is found.
Raises:	MatchTimeout if no match is found after timeout_secs seconds.

stbt.wait_for_motion¶

stbt.wait_for_motion(timeout_secs=10, consecutive_frames=None, noise_threshold=None, mask=None, region=Region.ALL, frames=None)¶

Search for motion in the device-under-test’s video stream.

“Motion” is difference in pixel values between two frames.

Parameters:	timeout_secs (int or float or None) – A timeout in seconds. This function will raise MotionTimeout if no motion is detected within this time. consecutive_frames (int or str) – Considers the video stream to have motion if there were differences between the specified number of consecutive frames. This can be: a positive integer value, or a string in the form “x/y”, where “x” is the number of frames with motion detected out of a sliding window of “y” frames. This defaults to “10/20”. You can override the global default value by setting consecutive_frames in the [motion] section of .stbt.conf. noise_threshold (float) – See detect_motion. mask – See detect_motion. region – See detect_motion. frames – See detect_motion.
Returns:	MotionResult when motion is detected. The MotionResult’s time and frame attributes correspond to the first frame in which motion was detected.
Raises:	MotionTimeout if no motion is detected after timeout_secs seconds.

Added in v28: The region parameter.

Added in v29: The frames parameter.

stbt.wait_for_transition_to_end¶

stbt.wait_for_transition_to_end(initial_frame=None, region=stbt.Region.ALL, mask=None, timeout_secs=10, stable_secs=1)¶

Wait for the screen to stop changing.

In most cases you should use press_and_wait to measure a complete transition, but if you need to measure several points during a single transition you can use wait_for_transition_to_end as the last measurement. For example:

stbt.press("KEY_OK")  # Launch my app
press_time = time.time()
m = stbt.wait_for_match("my-app-home-screen.png")
time_to_first_frame = m.time - press_time
end = wait_for_transition_to_end(m.frame)
time_to_fully_populated = end.end_time - press_time

Parameters:	initial_frame (stbt.Frame) – The frame of video when the transition started. If None, we’ll pull a new frame from the device under test. region – See press_and_wait. mask – See press_and_wait. timeout_secs – See press_and_wait. stable_secs – See press_and_wait.
Returns:	See press_and_wait.

stbt.wait_until¶

stbt.wait_until(callable_, timeout_secs=10, interval_secs=0, predicate=None, stable_secs=0)¶

Wait until a condition becomes true, or until a timeout.

Calls callable_ repeatedly (with a delay of interval_secs seconds between successive calls) until it succeeds (that is, it returns a truthy value) or until timeout_secs seconds have passed.

Parameters:

callable – any Python callable (such as a function or a lambda expression) with no arguments. timeout_secs (int or float, in seconds) – After this timeout elapses, wait_until will return the last value that callable_ returned, even if it’s falsey. interval_secs (int or float, in seconds) – Delay between successive invocations of callable_. predicate – A function that takes a single value. It will be given the return value from callable_. The return value of this function will then be used to determine truthiness. If the predicate test succeeds, wait_until will still return the original value from callable_, not the predicate value. stable_secs (int or float, in seconds) – Wait for callable_’s return value to remain the same (as determined by ==) for this duration before returning. If predicate is also given, the values returned from predicate will be compared.

Returns:

The return value from callable_ (which will be truthy if it succeeded, or falsey if wait_until timed out). If the value was truthy when the timeout was reached but it failed the predicate or stable_secs conditions (if any) then wait_until returns None.

After you send a remote-control signal to the device-under-test it usually takes a few frames to react, so a test script like this would probably fail:

press("KEY_EPG")
assert match("guide.png")

Instead, use this:

press("KEY_EPG")
assert wait_until(lambda: match("guide.png"))

Note that instead of the above assert wait_until(...) you could use wait_for_match("guide.png"). wait_until is a generic solution that also works with stbt’s other functions, like match_text and is_screen_black.

wait_until allows composing more complex conditions, such as:

# Wait until something disappears:
assert wait_until(lambda: not match("xyz.png"))

# Assert that something doesn't appear within 10 seconds:
assert not wait_until(lambda: match("xyz.png"))

# Assert that two images are present at the same time:
assert wait_until(lambda: match("a.png") and match("b.png"))

# Wait but don't raise an exception:
if not wait_until(lambda: match("xyz.png")):
    do_something_else()

# Wait for a menu selection to change. Here ``Menu`` is a `FrameObject`
# with a property called `selection` that returns a string with the
# name of the currently-selected menu item:
# The return value (``menu``) is an instance of ``Menu``.
menu = wait_until(Menu, predicate=lambda x: x.selection == "Home")

# Wait for a match to stabilise position, returning the first stable
# match. Used in performance measurements, for example to wait for a
# selection highlight to finish moving:
press("KEY_DOWN")
start_time = time.time()
match_result = wait_until(lambda: stbt.match("selection.png"),
                          predicate=lambda x: x and x.region,
                          stable_secs=2)
assert match_result
end_time = match_result.time  # this is the first stable frame
print("Transition took %s seconds" % (end_time - start_time))

Added in v28: The predicate and stable_secs parameters.

v31¶

19 September 2019.

Major new features:

• Supports test-scripts written in Python 3. Python 2 is still supported, too. When upgrading your test-pack to v31 you will need to specify a Python version in your test-pack’s .stbt.conf file like this:

  [test_pack]
  stbt_version = 31
  python_version = 3
  

  Valid values are 2 or 3. The test-run environment is Ubuntu 18.04, so you get Python 2.7 or 3.6.

  We recommend Python 3 for all new test-packs. For large existing test-packs we will continue to support Python 2 until all our large customers have migrated.

• stbt.Keyboard: New API for testing & navigating on-screen keyboards.

• stbt.Grid: New API for describing grid-like layouts.

Minor additions, bugfixes & improvements:

• stbt.match: Fix false negative when using MatchMethod.SQDIFF and a reference image that is mostly transparent except around the edges (for example to find a “highlight” or “selection” around some dynamic content).
• stbt.match: Improve error message when you give it an explicit region that is smaller than the reference image.
• stbt.ocr: New parameter char_whitelist. Useful when you’re reading text of a specific format, like the time from a clock, a serial number, or a passcode.
• stbt.press_and_wait: Ignore small moiré-like differences between frames (temporal dithering?) seen with Apple TV.
• stbt.press_and_wait: Draw motion bounding-box on output video (similar to stbt.wait_for_motion).
• stbt.press_and_wait: Add key attribute (the name of the key that was pressed) to the return value.
• stbt.Region: The static methods intersect and bounding_box will fail if called on an instance. That is, instead of calling self.intersect(other) you must call stbt.Region.intersect(self, other). Previously, if called on an instance it would silently return a wrong value.
• stbt.wait_for_motion: More sensitive to slow motion (such as a slow fade to black) by comparing against the last frame since significant differences were seen, instead of always comparing against the previous frame.
• stbt lint improvements:
  • New checker stbt-frame-object-get-frame: FrameObject properties must use self._frame, not stbt.get_frame().
  • New checker stbt-frame-object-property-press: FrameObject properties must not have side-effects that change the state of the device-under-test by calling stbt.press() or stbt.press_and_wait().
  • New checker stbt-assert-true: “assert True” has no effect.
  • Teach pylint that assert False is the same as raise AssertionError. This fixes incorrect behaviour of pylint’s “unreachable code” and “inconsistent return statements” checkers.

v30¶

25 February 2019.

Major new features:

• The test-job environment has been upgraded from Ubuntu 14.04 to Ubuntu 18.04. This means that any third-party packages that you use in your test-scripts will have been upgraded too.

• In particular, Tesseract (the engine used by stbt.ocr) has been upgraded to version 4.0. Overall the OCR accuracy should improve slightly, but there may be regressions or differences in behaviour in some cases.

• stbt.match supports transparency in the reference images: Transparent pixels in the reference image will be ignored when looking for a match within the video-frame. To use this feature your reference image must be a PNG with an alpha (transparency) channel. We only support fully-opaque or fully-transparent pixels: any pixels that aren’t fully opaque are treated as fully transparent.

  • stbt.load_image will include the image’s alpha (transparency) channel if it had any transparent pixels.

• stbt.match: Added new MatchMethod.SQDIFF, and made it the default match method. This works better and more consistently than MatchMethod.SQDIFF_NORMED (the previous default). SQIFF_NORMED doesn’t work at all for completely black images or images with transparency, and it exaggerates differences for dark images. The “certainty” number for the new SQDIFF method is still normalised so that it is a number between 0.0 and 1.0, but the normalisation no longer depends on how bright the images were, so the result is more consistent. The new default match_threshold is 0.98.

  • To make upgrading easier, we have configured .stbt.conf in existing customer’s test-packs to use the previous defaults.

• stbt.match: Inverted the meaning of confirm_threshold. Now numbers closer to 0 mean “less strict” and numbers closer to 1 mean “more strict”. This is for consistency with match_threshold. The default value has changed accordingly, from 0.3 to 0.7.

  If you were overriding the default value, you need to set the new value to (1 - previous_value), for example change 0.16 to 0.84.

• Removed compatibility flag global.use_old_threading_behaviour. This was introduced in v28, but nobody needed it. See release notes for v28 below for more information.

Minor additions, bugfixes & improvements:

• stbt.FrameObject: Add refresh method, used by navigation functions that modify the state of the device-under-test.

• stbt.match allows matching a greyscale reference image against a greyscale frame (for example if you’ve applied some custom pre-processing, such as edge detection, to both the frame & reference images).

• stbt.MatchParameters: The match_method and confirm_method can be specified as enums (stbt.MatchMethod and stbt.ConfirmMethod respectively). For example:

  stbt.MatchParameters(match_method=stbt.MatchMethod.SQDIFF,
                       confirm_method=stbt.ConfirmMethod.NONE)
  

  Passing the old string values is still supported for backwards compatibility.

• stbt.press now returns an object containing information about the keypress, including the start time & end time of the keypress signal. This is intended to help making performance measurements.

• stbt.press respects interpress_delay_secs if hold_secs is specified.

• stbt.Region: New methods dilate and erode to grow or shrink the region in all directions.

• stbt.Region.bounding_box and stbt.Region.intersect can take more than 2 regions.

• Roku HTTP control: Enforce 3 second timeout on all HTTP requests.

• stbt-lint improvements:

  • Uses pylint 1.8.3 (upgraded from 1.6.4).
  • New checker stbt-uncommitted-image: Detects when the filename given to stbt.match (and similar functions) exists on disk, but isn’t committed to git.
  • stbt-frame-object-missing-frame: Also checks for missing frame parameter when calling class constructors (not just class methods).
  • stbt-unused-return-value: Also checks that the return value of stbt.press_and_wait is used.
  • stbt-missing-image: Reports the full path to the missing image (relative to your test-pack root).
  • stbt-missing-image: Ignores filenames inside str.replace and re.sub.

v29¶

21 June 2018.

New features:

• Added APIs for testing audio: stbt.audio.get_rms_volume, stbt.audio.wait_for_volume_change, and stbt.audio.audio_chunks.
• Audio track is now included in the recorded videos.
• Added stbt.press_and_wait: Detection and frame-accurate measurement of animations and transitions.
• stbt.press accepts a new hold_secs parameter to hold a key down for the specified duration.
• Added stbt.pressing: A context manager that will hold a key down for as long as the code in the with block is executing.
• stbt.ocr and stbt.match_text: Added text_color_threshold parameter, to configure the threshold used with the text_color parameter.
• If your test-pack is a Python package (that is, it contains an __init__.py file in each directory under tests/) then relative imports from your test scripts will now work. This allows you to organise your tests into multiple directories easily.
• Stb-tester will determine the frame-rate of your device-under-test automatically; Stb-tester will capture at 25 or 30 frames per second, depending on the source frame-rate. Previously Stb-tester always captured at 25 fps. This fixes a source of sampling artifacts in performance measurements for devices that output 30 or 60 fps. You can restore the old behaviour by setting frame_rate = 25 in the [test_pack] section of .stbt.conf.

Breaking changes:

• stbt.is_screen_black returns an object with black and frame attributes, instead of a bool. This evaluates to truthy or falsey so this change is backwards compatible, unless you were explicitly comparing the result with == True or is True. This change was made so that you can get the exact frame that changed to (or from) black, for more precise performance measurements.

• stbt.frames returns an iterator of stbt.Frame objects, instead of an iterator of tuples (stbt.Frame, int). The second field of the tuple was a timestamp in nanoseconds; use the Frame’s time attribute (in seconds) instead.

  This function is rarely used so it shouldn’t affect most users. But if you were calling it like this:

  for frame, _ in stbt.frames():
  

  then you should change it to this:

  for frame in stbt.frames():
  

• Similarly, removed the deprecated timestamp attribute (nanoseconds) from stbt.MatchResult, stbt.TextMatchResult, and stbt.MotionResult. Use the time attribute instead (seconds).

Bug fixes & improvements:

• Fix AttributeError when using a stbt.FrameObject instance from multiple threads.
• stbt.press: Fix timing of the key-press visualisation in the recorded video. The key-press was appearing earlier than it was actually sent.
• stbt.Frame: Show timestamp & dimensions in repr output, instead of the verbose numpy repr.
• Debug logging: stbt.is_screen_black logs the result of its analysis, similar to stbt.match, stbt.ocr, etc.
• Debug logging: Only log 3 decimal places for frame timestamps. 60fps is 16ms per frame, so higher precision is misleading.
• Test-result artifacts: Screenshot & video thumbnail are saved correctly to the artifacts directory even if the test-script changed the current working directory with os.chdir.
• Web interface: Fix visualisation of the current line of code being executed, when running a test.

v28¶

6 February 2018.

Breaking changes:

• Passing region=None to stbt.ocr raises a TypeError instead of printing a deprecation warning. Use region=stbt.Region.ALL instead.

• Passing type_=bool to stbt.get_config now returns False for values of “0”, “false”, “off”, and “no” (ignoring case). Previously it would always return True for any non-empty value.

• stbt.get_frame and stbt.frames now return read-only frames for better performance. Use frame.copy() to get a writeable copy of the frame.

• stbt.get_frame is no longer guaranteed to return a new frame; it may return the same frame that the previous call to stbt.get_frame() returned. This may have subtle effects on the timing of existing test-scripts. Functions that depend on this behaviour should be refactored to use the stbt.frames iterator function instead.

  The benefit is that you can now call stbt.get_frame() from multiple threads. Also, code like wait_until(lambda: match('a.png') or match('b.png')) can run faster as the second match will no longer block waiting for a new frame.

  If this change causes you problems, you can add the following to your .stbt.conf file to restore the old behaviour:

  [global]
  use_old_threading_behaviour = true
  

New features:

• Multithreading support: stbt functions like wait_for_match can now be used from multiple threads simultaneously. Each call to stbt.frames returns an independent iterator that can be used concurrently.
• New function stbt.crop to crop a region from a video-frame.
• New function stbt.load_image to load an image from disk, using the same path lookup algorithm used by wait_for_match and friends.
• ocr and match_text have a new parameter text_color. Specifying this can improve OCR results when tesseract’s default thresholding algorithm doesn’t detect the text, for example for light-colored text or text on a translucent overlay.
• The pre-processing performed by ocr and match_text can now be disabled by passing upscale=False. This is useful if you want to do your own pre-processing.
• The default lang (language) parameter to ocr and match_text is now configurable. Set lang in the [ocr] section of .stbt.conf.
• press_until_match, detect_motion, wait_for_motion, and is_screen_black can take a new region parameter.
• The mask parameter to detect_motion, wait_for_motion, and is_screen_black can be an OpenCV image (previously it could only be a filename). This makes it easier to construct masks programmatically using numpy.
• The MotionResult object returned by detect_motion and wait_for_motion includes the video-frame that was analysed (in the new frame attribute). This allows you to perform additional analysis – for example if there was no motion is the frame black?
• wait_until has two new parameters: predicate and stable_secs. Together they allow waiting for something to stabilise (for example to wait for the position of a moving selection to stop moving).
• wait_until will try one last time after the timeout is reached. This allows you to use a short timeout with operations that can take a long time.
• The is_visible property of stbt.FrameObject subclasses can now call other public properties. Furthermore, the value of is_visible is always a bool, so you don’t have to remember to cast it to bool in your subclass’s implementation.
• The video recording in the test-results now runs at the full frame-rate rather than slowing down during wait_for_match. As a side-effect, if the image processing is particularly slow the annotations won’t appear on the output video. Apart from that caveat, annotations now appear if you got the frame using stbt.get_frame (previously the annotations only appeared if the frames came from a stbt.frames iterator). “Annotations” means the yellow & red rectangles showing the current match or motion region, or text added with draw_text.

v27¶

Minimum version supported by the Stb-tester Platform.