Sikuli Script: Troubleshooting Problems with Scripts

One of Sikuli's appealing features is its relatively straightforward concept; ironically, that can make it all the more puzzling when things aren't working quite the way they should-- you can see your button right there on the screen, so why can't Sikuli?  This post has a couple of tips for new users for troubleshooting common script problems.

Note: as of this writing, the latest version of Sikuli is version 930, available in Windows as a set of files to be unzipped over the previous version, version 905.  Version 930 includes some significant fixes; if you're experiencing issues and using version 905, I'd recommend upgrading to version 930 as a first step.

If Sikuli can't find a graphic pattern:

1) Be sure to check any FindFailed error messages carefully.  I know, I know-- that may sound obvious, but when you're knee-deep in a scripting session and you've already worked through half a dozen "FindFailed" errors in the last hour, you may see the seventh pop up in Sikuli's log and immediately assume it's the same as the others.  A case in point from my own personal experience: after troubleshooting for some time to figure out why Sikuli wasn't finding a match for my graphic pattern, a closer inspection of the FindFailed error revealed that Sikuli wasn't looking for a graphic pattern at all-- for some reason it was looking for text (treating my pattern file's name as a string).  Among other things, a FindFailed error can also indicate a problem finding a png file in its expected location on disk.

2) If you're looking for the pattern in a defined region (as opposed to a search on the entire screen), make sure the region covers the intended area and fully contains the pattern you're looking for; one way to check this is by calling the highlight() method on your region-- e.g., my_defined_region.highlight(2). You can remove the call later on once you've finished debugging.

3) Don't be stingy with wait times.  Sikuli can interact with an application much faster than a human, and that's not always a good thing.  In some cases it may not find a pattern simply because it hasn't had time to draw completely (in response to some previous action).  Use a simple wait() statement to give the screen, region, etc. more time to draw before proceeding to the next step.

If Sikuli incorrectly identifies a match:

1) A mis-click near a pattern (close but not centered on it) may stem from re-drawing and or re-sizing a window or components in a window. Consider a scenario where Sikuli is waiting for a button to appear onscreen; as soon as it appears it registers the match.  Subsequently, the window finishes drawing, during which the placement of the button is adjusted to make room for other components.  Sikuli doesn't recognize this change in the button's location-- as far as it's concerned, the button is still where its pattern was first detected, and that's where Sikuli will try to interact with it.  Possible solutions in this case include inserting a fixed (designated in seconds) wait() statement or using another pattern as a cue to Sikuli that the screen has finished drawing.

2) If Sikuli keeps matching a pattern to items other than the one you're actually trying to isolate, first make sure your pattern is "focused"-- you generally want to avoid any nondescript surrounding white space that doesn't help Sikuli differentiate it from similar regions.  You can also increase the similarity setting for the pattern to require a closer match.  With your script loaded in Sikuli, bring up the application screen where a match for your pattern should be found and click on the pattern in your script, launching the Pattern Settings dialog.  The Matching Preview tab shows the current screen, highlighting any regions that match your pattern-- if you adjust the Similarity slider, the preview automatically updates to reflect the new value.  Yet another solution: limiting your search to a specific region around your target, excluding other potential matches outside the region.


The Matching Preview tab of the Pattern Settings dialog

For scripting issues not necessarily related to finding pattern matches, the popup() method can be used as a poor man's break point for debugging.  For example, you can use it to pause your script and print out the value of a counter variable in a for loop, a value evaluated in an if statement, etc.  The method can be removed from your final script once you've finished troubleshooting.

Finally, when all else fails or you're seeing completely inexplicable issues, save your work, shut down Sikuli, and check for any orphaned processes (in Windows, these appear as javaw.exe instances) before restarting.  I've seen scenarios where a script didn't seem to shut down properly (particularly when execution was interrupted with Alt + Shift + C) and behavior becomes flaky and unpredictable when I attempt to re-run the script or run another script.  Clearing out these orphaned processes and re-starting Sikuli sometimes helps in these cases.

No comments:

Post a Comment

Please be respectful of others (myself included!) when posting comments. Unfortunately, I may not be able to address (or even read) all comments immediately, and I reserve the right to remove comments periodically to keep clutter to a minimum ("clean" posts that aren't disrespectful or off-topic should stay on the site for at least 30 days to give others a chance to read them). If you're looking for a solution to a particular issue, you're free to post your question here, but you may have better luck posting your question on the main forum belonging to your tool's home site (links to these are available on the navigation bar on the right).