Python API Overview#

Testcases are Python functions stored in the test-pack git repository under tests/*.py. The function name must begin with test_.

Example#

import stbt

# You can import your own helper libraries from the test-pack.
import dialogues


def test_that_pressing_EPG_opens_the_guide():
    # We recommend starting each testcase with setup steps so that
    # the testcase can be run no matter what state the device-under-
    # test is in. Note that you can call other Python functions
    # defined elsewhere in your test-pack.
    if dialogues.modal_dialogue_is_up():
        dialogues.close_modal_dialogue()

    # Send an infrared keypress:
    stbt.press("KEY_EPG")

    # Verify that the device-under-test has reacted appropriately:
    stbt.wait_for_match("guide.png")

Controlling the system-under-test#

Remote control#

stbt.press: Send the specified key-press to the system-under-test.
stbt.pressing: Press and hold a key until some other condition is met.
stbt.press_and_wait: Press a key, then wait for the screen to change, then wait for it to stop changing.
stbt.press_until_match: Call press as many times as necessary to find the specified image.

Network-based protocols#

Some devices (such as the Roku and some Smart TVs) can be controlled via HTTP or other network protocols. You can use any Python networking library to make network requests to such devices (to install third-party Python libraries see Customising the test-run environment). We recommend the Python requests library, which is already installed.

Verifying the system-under-test’s behaviour#

Searching for an image#

stbt.wait_for_match: Search for the specified image in the video stream; raise stbt.MatchTimeout if not found within a certain timeout.
stbt.match: Search for the specified image in a single video frame; return a truthy/falsey stbt.MatchResult.
stbt.match_all: Search for all instances of the specified image in a single video frame.

Use stbt.match with assert and stbt.wait_until for a more flexible alternative to stbt.wait_for_match. For example, to wait for an image to disappear:

stbt.press("KEY_CLOSE")
assert wait_until(lambda: not stbt.match("guide.png"))

Searching for text using OCR (optical character recognition)#

stbt.match_text: Search for the specified text in a single video frame; return a truthy/falsey stbt.TextMatchResult.
stbt.ocr: Read the text present in a video frame.

Searching for motion#

stbt.wait_for_motion: Search for motion in the video stream; raise stbt.MotionTimeout if no motion is found within a certain timeout.
stbt.detect_motion: Generator that yields a sequence of one stbt.MotionResult for each frame processed from the system-under-test, indicating whether any motion was detected.

Miscellaneous video APIs#

stbt.is_screen_black: Check whether the video frame is completely black.

Audio APIs#

stbt.get_rms_volume: Calculate the average RMS volume over a given duration.
stbt.wait_for_volume_change: Wait for changes in the RMS audio volume. Can detect the start of content playback or unmuting; bleeps or clicks while navigating the UI; or beeps in an A/V sync video.
stbt.audio_chunks: Low-level API to get raw audio samples for custom analysis.

Custom image processing#

Stb-tester can give you raw video frames for you to do your own image processing with OpenCV’s “cv2” Python API. Stb-tester’s video frames are numpy.ndarray objects, which is the same format that OpenCV uses.

stbt.frames: Generator that yields frames captured from the system-under-test. Example usage: for frame in stbt.frames(): ...
stbt.get_frame: Return the latest video frame.
stbt.load_image: Load an image from disk.
stbt.crop: Crop an image using a stbt.Region.

To save a frame to disk, use cv2.imwrite. Note that any file you write to the current working directory will appear as an artifact in the test-run results.

Logging#

stbt.draw_text: Write the specified text on this test-run’s video recording.

Anything you write to stdout or stderr appears in the test-run’s logfile in stb-tester’s test-results viewer.

Metrics#

For some customers we run Prometheus and Grafana on your Stb-tester Portal. (Prometheus is an open-source time-series database for metrics; Grafana is an open-source dashboard & reporting tool driven by the data in Prometheus.) If this is enabled on your Portal, you can log metrics to Prometheus using the following APIs:

Utilities#

stbt.as_precondition: Mark test failures as test errors in some parts of your testcase.
stbt.FrameObject: Base class for user-defined Page Objects, which are a layer of abstraction between your testcases and the stbt image processing APIs.
stbt.get_config: Read a value from the test-pack’s Configuration files.
stbt.wait_until: Wait until a condition becomes true, or until a timeout.

Exceptions#

If your testcase raises one of the following exceptions, it is considered a test failure:

AssertionError (raised by Python’s assert statement).
stbt.MatchTimeout (raised by stbt.wait_for_match).
stbt.MotionTimeout (raised by stbt.wait_for_motion).
stbt.UITestFailure: Inherit from this if you need to define your own test-failure exceptions.

Any other exception is considered a test error. For some examples see Investigating intermittent bugs.