Sikuli IDE: Using Pattern Objects

If you go back to the script illustrated in the last post and look at the code for click() or wait() calls in IDLE or some other IDE for Python (other than Sikuli's own editor), you'll see something like the following:

...
wait("FileEditForm.png",5)
...
click("1351739445615.png")

Beneath all the source images in our script (as displayed in Sikuli's IDE) are strings matching the image file names.  When rendering the script in its IDE, Sikuli interprets these strings as instances of its Pattern object, replacing them with the images they represent.  It's important to recognize that we are not limited to using Pattern objects within click() and wait() calls directly-- we can manipulate them programmatically to make scripts more flexible and/or efficient.

To illustrate, let's say we have an app that looks like this:


When the user clicks on any of the color buttons along the bottom of the app form, the large "Color" text changes to match that color.  A straightforward way of writing a test script would be to write four pairs of click() and exists() methods, with each click() on one of the four buttons followed by an exists() check on the "Color" text (to verify it's displaying the right color).  But keeping Pattern objects in mind we could also organize our script like this:


A Sikuli script with Pattern objects organized in a list
In this script, the Pattern objects (images) are organized into a list of tuple pairs; the first image in each pair is the button being clicked, and the second is the "Color" text as we expect it to appear when each button is clicked.  We can use the image list with a for loop to check all four buttons. Using a list like this opens up other possibilities, too-- for example, we could easily step through it randomly instead of following a set order. Obviously, this is an oversimplified example, but it still reflects a fundamental idea behind UI testing that can be applied to more complex applications: interacting with one UI element (via a click, for example), then verifying expected behavior by confirming the existence (or non-existence) of some associated visual "cue".

A few words about some "gotchas" when working with source images: going back to the example in the first paragraph about file name strings, there's no path information necessary with this particular example because the files are located in the same directory as the Sikuli script.  Of course, if you wanted to put your source images in a different directory, you'd have to add corresponding path information to the file name string.  Be aware that as of build 905 of Sikuli X, moving the images out of the main .sikuli folder for your script can also result in problems with rendering them properly in Sikuli's IDE-- you'll see some errors appear in the Message pane at the bottom of the IDE when loading your script, and where you'd expect to see images you may see the underlying strings instead. Here's what my image list looks like in Sikuli's IDE after moving the images to a sub-directory in the .sikuli folder called "Imgs" (and modifying the script accordingly):

image_list = [("\\Imgs\\EE.png","\\Imgs\\Color.png"),
        ("\\Imgs\\1354395248414.png","\\Imgs\\C0l0r.png"),
        ("\\Imgs\\1354395299725.png","\\Imgs\\C0l0r-1.png"),
        ("\\Imgs\\1354395332765.png","\\Imgs\\C0l0r-2.png")]

The script still runs successfully, but the images themselves are not loaded and displayed in the IDE.

One alternative way to deal with source images located in directories other than the .sikuli folder is the addImagePath() method.  Sikuli maintains an image search path-- a list of directories that it will search when a given image can't be found in the active .sikuli folder.  So for example, if you moved an image into a directory called "C:\Temp\Images", you could just add the directory to the image search path using the following code:

imgPath = list(getImagePath())
if 'C:\\Temp\\Images' not in imgPath:
    addImagePath('C:\\Temp\\Images')

Subsequently, the full path for any images in that directory would not have to be specified-- you could just specify the image file name by itself; Sikuli would automatically include the "C:\Temp\Images" directory when looking for the image.  Note that while this method does save some time typing out paths when specifying images, it may still result in the same problems explained above when using non-.sikuli folders for images.