Stb-tester v33 supports running your tests with pytest, a popular Python test-runner. Pytest provides powerful functionality such as assertion introspection and fixtures.

Assertion introspection

When an assert in your test fails, pytest prints additional information about the value that triggered the assertion. Without pytest the output looks like this:

    assert menu.selection == "Channels"
AssertionError

Pytest’s output looks like this:

>       assert menu.selection == "Channels"
E       AssertionError: assert 'Settings' == 'Channels'
E         - Channels
E         + Settings
tests/btsport/demo.py:13: AssertionError

Much more helpful!

To learn more, see pytest’s documentation about asserting.

Fixtures

Pytest’s fixtures allow you to define setup and teardown code that can be re-used in many tests. For example, the following fixture captures logs from the device-under-test while the test is running:

@pytest.fixture
def logs():
    start_capturing_logs("dut.log")
    try:
        yield
    finally:
        stop_capturing_logs()

(I’ll leave the implementation of start_capturing_logs() and stop_capturing_logs() up to you. 😁)

To use this fixture in any test, just pass it as an argument to the test:

def test_menu_navigation(logs):
    page = launch_my_app()
    page = page.navigate_to("Settings")
    assert page.selection == "Settings"

In the fixture, code before the yield will run before the test; code in the finally block after the yield will run after the test (even if the test raised an exception).

To learn more, see About fixtures in the pytest documentation.

Configuration

In Stb-tester v33, using pytest as the test-runner is optional. From Stb-tester v34, pytest is the default (and only) test-runner.

To enable pytest on v33, use the following configuration in your test-pack’s .stbt.conf file (you don’t need this on v34 or newer):

[test_pack]
stbt_version = 33
pytest = true

Notes for pytest experts

We run each test-case in a separate pytest process. This means that there’s no difference between “session” fixtures and “function” fixtures — each test function is run in a separate session.

Support for parametrized tests is limited: All parametrized variants are run as a single test-run, so you get a single test-result (a single video, single logfile, and single pass/fail outcome) for all of them.

Both of these limitations may be removed in a future Stb-tester release.