Automating SoapUI with Testrunner

In the SoapUI examples I've provided so far, test suites have been run through the UI.  At some point, however, you'll probably want to take your automation to the next step, running suites unattended and reviewing test results later.  We'll take a look at those topics over the next few posts, starting with SoapUI's testrunner.

Testrunner.bat typically resides in the bin sub-directory of SoapUI's program folder-- e.g., C:\Program Files (x86)\SmartBear\SoapUI-5.0.0\bin.  It allows you to run a SoapUI test suite from the command line with a variety of switches to set configuration options-- quite an extensive variety, in fact-- you can find the full list on the testrunner page on SoapUI's site.  Note that some of the switches listed here are only available in the Pro version of SoapUI.

Here's an example of a typical command sequence illustrating some of the more common switches you'll probably end up using:

"C:\Program Files (x86)\SmartBear\SoapUI-5.0.0\bin\testrunner.bat" -s"periodictableSoap TestSuite" -fC:\Temp\TestOutput -I -j -M C:\My Documents\MyPeriodicTableProject.xml

Going through each element of the sequence:

- "C:\Program Files (x86)\SmartBear\SoapUI-5.0.0\bin\testrunner.bat" : This is simply the path to testrunner.bat.  For those of you unfamiliar with using the command line, note the path is enclosed in quotations because it contains spaces.  Spaces normally separate elements of a command sequence; surrounding the path in quotations ensures the full path is treated as a single element.  Also, the full path is unnecessary if you've changed the working directory to the bin directory (i.e., you'd only have to specify "testrunner.bat"-- you can see this in the SoapUI TestRunner dialog below).

- -s"periodictableSoap TestSuite" : The -s switch precedes the name of the test suite within your project that you'd like to run-- as with the path to testrunner.bat, the name of the suite is enclosed in parentheses because it contains a space.

- -fC:\Temp\TestOutput : The -f switch precedes the path to the directory where test results will be saved.

- -I : This switch instructs testrunner to continue running even if an error is encountered.  Switches are case-sensitive (-i has a different meaning).

- -j : While SoapUI Pro supports a number of additional switches to generate different types of reports, the -j switch is available in the free version to generate JUnit-style xml reports.  These reports are saved to the directory specified with the -f switch above.

- -M : Separate from the JUnit-style report enabled by the -j switch, the -M switch saves a run log report to the results directory.

- C:\My Documents\MyPeriodicTableProject.xml : Finally, we specify the path to the project containing the suite to be run (no switch needed here).

Until you're comfortable with all of its switches and constructing a command sequence from scratch-- or just as a matter of preference-- you can also launch testrunner via the UI to let SoapUI generate a command sequence for you.  In the workspace tree, right-click on the name of the test suite for which you'd like to generate your command sequence (or the test project if you'd like to run multiple test suites in a given project) and select "Launch TestRunner" to bring up the configuration dialog.

Select your options from the various tabs in the dialog (the Basic and Reports tabs contain the more commonly used settings) and click the Launch button.  A command sequence corresponding to the selected configuration options is displayed near the top of the testrunner output (highlighted in the screenshot below).

Once you have this command line sequence it opens up a wide range of possibilities: you can incorporate it into your own batch files or Powershell scripts, integrate SoapUI test runs with an automation solution like Jenkins (incidentally, you can also use Jenkins to publish JUnit reports), schedule a run with Windows Task Scheduler, etc.

As mentioned above, the free version of SoapUI supports JUnit-style reports out of the box; however, it's possible to create some home-brewed reports using Groovy scripting.  We'll take a look at writing results to XLS files using the POI API in the next post.