soapUI: A Simple Data-Driven Testing Technique

Note: If you experience any issues with webservicex.net or the periodic table service used in this post, or would just like to see another example, I've provided an alternative post covering pretty much the same material with a different service here.

For this post, I'd like to walk through an example project based on the periodic table service used in previous posts to illustrate the use of Groovy script.  You can download the project and a data file used by the project here. After unzipping the files, open soapUI and use File - Import Project to pull the project (xml file) into your workspace. You can put the TestData.csv file anywhere you want, really, but its default location in the script is a directory called C:\PerTableData-- you'll have to modify one line in the project if you want to put it somewhere else (note: to see one way to modify this project to use an Excel data source instead of CSV, see my post here).  Here's what the project tree should look like:


Example project tree
The four operations provided by the service (GetAtomicNumber, GetAtomicWeight, GetElementSymbol, and GetAtoms) are grouped into two separate test cases.  The GetAtoms operation has no request parameters and is in its own test case.  The three remaining operations are grouped together in the ElementSpecificRequests test case for several reasons.  All of them use a single element name as their only request parameter, and there's some overlap between the data returned by the operations (the GetAtomicNumber request returns atomic weights and symbols, which are also returned by the GetAtomicWeight and GetElementSymbol requests, respectively).  Ideally, of course, we'd like to test these three "element specific" operations using more than one symbol, so they're all candidates for data looping (repeating the test requests with different input parameters), too.

Our strategy for data looping is as follows:

1) Create a data file that provides request input parameters (element names, in this case) AND the expected response data for each input so we can perform content assertions.
2) Read in a single line of the file and populate test case properties with the data.
3) Perform our test requests and assertions using the test case properties we populated from the file.
4) Repeat steps 2 and 3 until we reach the end of our file (i.e., no more lines to read in).
5) End the test case.

The TestData.csv file is our source file for step 1.  I personally prefer using csv files for this purpose because they generally have a clear delimiter and they're easy to create and maintain in spreadsheet programs like Excel.  Here are the contents of TestData.csv; each line consists of an element's name, atomic number, symbol, atomic weight, and boiling point (you may notice some errors-- these are there to simulate test step "fails" later on):

Hydrogen,1,H,1.00797,20.400000000000002
Carbon,6,C,12.0115,51000
Oxygen,8,O,15.9994,90.2
Gold,79,Gd,196.967,3239
Uranium,92,U,238.03,4091

For data looping there's also a test case option that we want to change.  If you launch the Options dialog (right-click on a test case in the navigator and select Options from the context menu), you'll see that there's an option called "Abort on Error".  We want to make sure this is unchecked-- otherwise, execution will stop and move on to the next test case the first time an error occurs; obviously, we want the test case to continue looping even if an error occurs during one loop.

The Groovy script to start reading our csv file isn't contained in a test step-- instead, it's in the ElementSpecificRequests test case's Setup script.   At the test suite and test case levels, Setup and TearDown scripts are available-- the Setup script is run prior to any of the test steps of the test case (or prior to any of the test cases when its at the suite level), and the TearDown script is run after the last test step (or test case for suite level TearDown scripts) has completed.


The Test Case Editor with the Setup Script shown (note the button to show the TearDown Script, too)
Here's our Setup Script:

//Create a new filereader object, using the context variable so it can be used between test components
context.fileReader = new BufferedReader(new FileReader("C:\\PerTableData\\TestData.csv"))
//Read in the first line of the data file
firstLine = context.fileReader.readLine()
//Split the first line into a string array and assign the array elements to various test case properties
String[] propData = firstLine.split(",")
testCase.setPropertyValue("ElName",propData[0])
testCase.setPropertyValue("AtNum",propData[1])
testCase.setPropertyValue("Symbol",propData[2])
testCase.setPropertyValue("AtWeight",propData[3])
testCase.setPropertyValue("BoilPoint",propData[4])
//Rename request test steps for readability in the log; append the element name to the test step names
testCase.getTestStepAt(0).setName("GetAtomicNumber-" + propData[0])
testCase.getTestStepAt(1).setName("GetAtomicWeight-" + propData[0])
testCase.getTestStepAt(2).setName("GetElementySymbol-" + propData[0])

The first (non-comment) line creates a FileReader object to read our data input file (if you put the csv file into a location other than C:\PerTableData, you need to modify this line accordingly).  Note that our fileReader object is created as a property of the context variable.  The context variable is readily available in Setup and Teardown Scripts-- no need to declare it or otherwise initialize it.  It provides a way to share objects, variables, etc., across the relevant run context.  In this case, the fileReader object will be available across the test steps of our test case-- handy because we'll need to keep track of where we are in our file as we read in each line.

In the remaining lines of the script we read in the first line of our file (line 4), split the line into a string array (line 6), and then assign the elements of the array to our test case properties (lines 7 - 11) using setPropertyValue(), a method of the testCase object.  In lines 13 - 15 we're actually renaming the test steps by appending element names to their default names-- this is not necessary, really, but helpful for reading the run log; otherwise, we'd see identical test step names for each iteration.  Note the use of the testCase variable in lines 7 - 15; like the context variable, this is automatically available for use in the Setup Script.

After our Setup Script runs, our test requests are sent and verified against content assertions.  We use property expansions with our test case properties for request parameters and assertion criteria:

GetAtomicNumber test request with an assertion-- note the property expansions used for the ElementName element in the request and the AtomicNumber element in the assertion
So after reading in the first line of our data file, the GetAtomicNumber test request gets sent using "Hydrogen" as the ElementName; the response is checked to see if it contains "1".

After our test requests, the next step in the test case is the ReadNextLine Groovy test step.  Here's the script contained in this test step:

/*Read in the next line of the file
  We can use the same fileReader created in the Setup script because it
  was assigned to the context variable.*/
nextLine = context.fileReader.readLine()
/*If the end of the file hasn't been reached (nextLine does NOT equal null)
  split the line and assign new property values, rename test request steps,
  and go back to the first test request step*/
if(nextLine != null){
 String[] propData = nextLine.split(",")
 curTC = testRunner.testCase
 curTC.setPropertyValue("ElName",propData[0])
 curTC.setPropertyValue("AtNum",propData[1])
 curTC.setPropertyValue("Symbol",propData[2])
 curTC.setPropertyValue("AtWeight",propData[3])
 curTC.setPropertyValue("BoilPoint",propData[4])
 curTC.getTestStepAt(0).setName("GetAtomicNumber-" + propData[0])
 curTC.getTestStepAt(1).setName("GetAtomicWeight-" + propData[0])
 curTC.getTestStepAt(2).setName("GetElementSymbol-" + propData[0])
 testRunner.gotoStep(0)
}

Most of this should look pretty familiar-- mostly, we're doing the same things we did in our Setup script.  In line 4 we're reading in the next line in our data file.  If the next line doesn't equal null (which is what would be returned if we had reached the end of the file), we go through the same steps we went through before: splitting the line up into a string array (line 9), assigning the elements of that array to different test case properties (lines 11 - 15), and re-naming the request test steps (lines 16 - 18).

Two new things: the last line of the script sends test case execution back to the first step in our test case, the GetAtomicNumber request.  Also note the use of the testRunner variable.  This is another one of those variables like the context variable that's ready to use from the start.  The testRunner variable is important in soapUI scripting as it's used as an entry point to get to other test objects and control execution.  Case in point: the testCase variable that was available as a "built-in" variable in our Setup script is not immediately available in our Groovy test step.  Instead, in line 10 we use our testRunner variable to get a reference to the test case.  Once we have the test case, we can get its test steps and their properties and methods, and so on.

So execution passes back to the first test step and our test requests are re-run, this time using property values read in from the second line of our data file for request parameters and assertion criteria.  This loop continues until we reach the end of the file-- when this happens, the next line read in is null, so the if statement in line 8 of the ReadNextLine Groovy step fails and execution is never passed back to the first test step.

Finally, before the test case ends its TearDown script is run:

//Cleanup: rename test steps to their generic names and close the file reader
testCase.getTestStepAt(0).setName("GetAtomicNumber")
testCase.getTestStepAt(1).setName("GetAtomicWeight")
testCase.getTestStepAt(2).setName("GetElementSymbol")
context.fileReader.close()

In this script we're just renaming the request steps back to their default names and in the final line closing our data file for reading.

Try running the test suite-- double-click on the test suite node in the navigator to launch the TestSuite Editor, then click the Run button in the editor window.  You should see the progress bar for the ElementSpecificRequests test case fill up multiple times, turning red after an error is encountered.  Once the test cases have completed running, open up the TestSuite Log.


The TestSuite Log for our periodic table test suite (with run results)
We can see right away that something failed for the GetAtomicNumber test request-- and because we were modifying the test step names as we went, we can see it occurred for the carbon element.  If you want to see more information about the failure, you can simply click on the error to launch the Message Viewer.  Here you can see the request and response XML (including the values resulting from property expansions) to get a better idea of where the problem may be.  In this case, when we compare the response with our error above we can see that our assertion was expecting a boiling point of 51000 but our response returned a value of 5100.


Message Viewer response
That illustrates one way to perform data looping with the soapUI free version.  It should be noted that the commercial version of soapUI has features to make data looping much easier with less scripting required.  If data looping is a feature you intend to use heavily, it may be worth it to weigh the cost of the commercial version against potential savings in time spent on script development and maintenance.