A better way to organise your testcases
31 Mar 2015.
Up until now, stb-tester testcases were Python scripts with one testcase per file, and you referred to your testcases by the filename.
When we were designing the stb-tester ONE we wanted a more expressive way of declaring and organising testcases. We took inspiration from standard Python testing frameworks like nose and pytest, where testcases are specified as Python functions:
With the stb-tester v22 release, this functionality is also available to the
stbt run and
stbt batch run command-line tools:
Organising your testcases in this way has several advantages:
It encourages smaller, more descriptive testcases.
It makes it easier to factor out and share common code between testcases.
You can attach documentation or metadata to testcases (using docstrings, nose’s attr decorator, etc.) which can be queried by importing the test file and inspecting the functions.
For example, you can attach metadata like “this testcase validates requirement ABC” or “this is a regression test for defect XYZ”; and you can write post-run scripts to update your defect-tracker after each testrun.
It brings stb-tester in line with standard Python testing frameworks like nose and pytest, which your Python programmers are already familiar with.
Drawing further inspiration from nose and pytest, stb-tester v22 also
allows you to use
assert in your testcases. We have also added a utility
function called wait_until that allows a very flexible and compact way of
expressing your testcases. See also this older blog post
where we had discussed the design of
wait_until as a way of making the
stb-tester API more composable.