I have written a Page Object for the “Search” page in the YouTube app on Apple TV. My Page Object has a property that reads the search term using OCR, but the OCR is mis-reading some words. Let’s investigate!

The page looks like this:

My Page Object’s search_text property reads the search term that the user has typed using the on-screen keyboard. First we look for the magnifying-glass icon, to determine the region where the text should appear:

Then we read the text to the right of the icon. Here is the code of my Page Object property that does this:

class Search(stbt.FrameObject):
    # 'is_visible' and other properties not shown

    @property
    def search_text(self):
        icon = stbt.match("search-icon.png", frame=self._frame)
        if icon:
            return stbt.ocr(
                frame=self._frame,
                region=icon.region.right_of(width=240))

One of my tests failed because the text didn’t match the value I was expecting. First we save and commit a screenshot under selftest/screenshots/. (I recommend that you organise the screenshots in subfolders according to device, application, and Page Object, for example selftest/screenshots/appletv/youtube/keyboard/peppa pig.png.)

Then we can debug the Page Object against this screenshot in Stb-tester’s Object Repository:

In the Object Repository, your Page Object’s properties are run against the committed screenshots, instead of a live video-frame from the device under test. Here we can see that the Page Object’s search_text property is reading “Denna piq” instead of “peppa pig”. To investigate, let’s click on the property’s drop-down menu and select Debug:

This will show detailed debug for each of the Stb-tester APIs that are called to evaluate this property (stbt.match to find the magnifying-glass icon and stbt.ocr to read the text). Now the problem is obvious: The region that we are giving to stbt.ocr is chopping off the bottom of the text!

The solution is to extend the region downward by a few pixels so that it includes the descenders in the text:

     @property
     def search_text(self):
         icon = stbt.match("search-icon.png", frame=self._frame)
         if icon:
             return stbt.ocr(
                 frame=self._frame,
-                region=icon.region.right_of(width=240))
+                region=icon.region.right_of(width=240).extend(bottom=5))

Now we can see in the object repository that search_text is correct:

Tip: When you’re trying out code changes to debug an issue like this, you can use stbt_rig.py snapshot to sync the code from your IDE to the Stb-tester Portal every time you press “Save”. See IDE Configuration in the Stb-tester manual.

To learn more about Page Objects, see Object Repository in the Stb-tester manual.