Training Test Automation Scripts for Dynamic Combat: Balance

In his second installment of the four-part Taekwondo-mation series, Dion Johnson teaches the art of balance in three straightforward steps. Balance requires equilibrium, and in test automation that translates to increasing the coverage of a single script without increasing the amount of code in the automated test. In this week's column, Dion explains how to increase the coverage by randomizing the data being tested.

Application behavior is dynamic, so it's time that we train our automated scripts to be more dynamic via the techniques provided in the automation martial arts known simply as "Taekwondo-mation." It comprises relatively simple techniques that can be employed without the implementation of an overly complex framework.

Taekwondo-mation, like many of the martial arts, places a focus on four key principles:

         1. Flexibility-Dynamically inputting data
         2. Balance-Inputting dynamic data
         3. Self-defense-Exception handling
         4. Strikes-Dynamic-path handling

I will discuss each of these principles in a separate article. The first article in this four-part series covers flexibility. In this second installment, I discuss Balance.

Achieving balanced requires being in a state of equilibrium. In life, balance is important for increasing the quality of life. In the martial arts, balance helps reduce the risk of injuries. In test automation, balance is important for both of these reasons as applicable to software testing.

In general, balance in testing typically refers to the adequacy of the test coverage in the test bed as a whole, which is more of a test-planning concern rather than something specific to automation. Balance in taekwondo-mation focuses on the test-automation script and on ways to subtly increase the coverage of a single script without increasing the amount of scripting inside of a single automated test. One way of achieving automated-test-script balance involves randomizing the data that is entered or selected in the application by the script. Other than the one-time creation of reusable functions that are responsible for randomizing the data, no additional scripting is required for each individual test script.

Before going any further, we must ask how randomizing data helps to achieve balance. The answer is in the fact that randomized data increases the application data coverage over time (cumulative coverage) within a script. This means that while each individual test execution may only traverse a single data scenario, the data scenario may change slightly during each test run, so multiple data scenarios are covered over time by the same script. To better understand, let's examine an old game known as block breaker. The object of the game is to break all the blocks at the top of the screen by using the paddle at the bottom of the screen to direct the ball into the blocks. Each time the ball hits the paddle it bounces up and breaks a different block.

Figure 1: Block Breaker Scenario 1

In figure 1, we see that the ball is about to break a block to the left of the screen.

Figure 2: Block Breaker Scenario 2

In figure 2, we see that by using the same paddle in a different scenario, the ball is about to break a block on the right side of the screen. What changed? The trajectory of the ball is what changed. So while just a single block is broken on each bounce of the ball, over time all of the blocks will be broken. This is what cumulative coverage is all about.

Balance in an automated script helps to increase quality of the application by increasing the test data coverage of that script over time. In addition, the potential of injuries sustained to a script are reduced. Injury to a script is when the script "breaks" during a test run. Limiting the dependencies the script has on specific data elements greatly reduces the chances of a script breaking. For example, figure 3 illustrates a dropdown list. While there may be a specific test with the objective of verifying the data in the list, all other tests may only care that a value is selected, without regard to what the value is. This may be an excellent candidate for random data selection.

Figure 3: Dropdown List

Randomly selecting data adds an exploratory aspect to the test execution that would normally only take place during manual test execution and may eventually uncover an issue with a corrupted data element. Also, if more than one data element in a test scenario is randomized, then the script will test multiple combinations of data selections over time. In addition, if the drop-down list has the ability to change dynamically, the changes will not have an adverse affect on the script, because the script will not be dependent on any specific data elements.

There are many types of data elements that may be randomized, such as dropdown lists, textboxes, links, radio button groups, etc. Regardless of the type of data element that is being randomized, the fundamental approach remains the same:

         1. Obtain a collection/array of objects or data.
         2. Identify the total number of elements in the collection/array.
         3. Use a random-number generator to select an element from the collection/array.

Step 1: Obtain a Collection/Array of Objects or Data
The first step of obtaining a collection/array of objects or data is the trickiest of all three steps, because the way this is accomplished will vary depending on the type of data element, the type of technology with which the application under test is built, and the automated test tool being used. In the event that the element is a text box used for entering a person's name, the array is not something that must be obtained from the application, but rather something that must be created in the script itself in a statement that resembles the following:

nameArray = Array(Tim, Jack, Diane, Christina)

Things get a little trickier when obtaining a collection of objects from the application itself, such as a group of links that appear in a table of search results. Table 1 reveals how a search results table may appear in an HTML application.

Table 1: Application Search Results Table

To open a detailed page for a given search result from this sample HTML table, an Edit link must be clicked. In order to cause a script to randomly click an Edit link, a collection of the page's Edit links first must be gathered. A collection is simply an array or group of similar objects. As previously stated, the automation tool being used will play a big part in how this is done, but it may involve a set of statements that first collects all the links on the screen, then narrows down that collection according to some property value that is common to only the Edit links in the search results table (i.e., htmlText = "Edit").

Step 2: Identify the Total Number of Elements in the Collection/Array
This step is typically the simplest of the three. Once the first step has been completed and the data or objects have been assigned to an array or collection, identifying the total number of elements in the array or collection is normally just a matter of using one of the standard methods that are provided with arrays or collections. This may resemble the following statement:

nameArrayCount = nameArray.length

Step 3: Use a Random Number Generator to Select an Element from the Collection/Array
This final step is what randomizes the data entry/selection process. Upon completing the second step, the element number can be entered as an argument to a random number generator function. Most scripting languages and or automation tools have some sort of randomizer that can be tweaked to function as a random number generator. Once the random number is generated, it can be used as an element index value used for entering/selecting data. For example, the random number generator will generate a random number from one to three when used on the collection of Edit links in table 1. Once the number is generated, it can be used in the statement that clicks a link. Such a statement might resemble the following:

clickLink("htmlText=Edit", "linkIndex=randomLinkIndex")

Conclusion
Random data entry can help ensure that a quality software product is being produced, can help to increase the robustness of an automated test script, and can be done via the three-step process explained in this article.

Stay tuned for Part III of the Taekwondo-mation series in which I will explore the principle of Self-defense (exception handling).

Read Training Test Automation Scripts for Dynamic Combat: Flexibility - Part I
Read Training Test Automation Scripts for Dynamic Combat: Self Defense - Part III
Read Training Test Automation Scripts for Dynamic Combat: Strikes - Part IV