Fixing the Brittleness Problem with GUI Tests

My team started using UI test automation around 2010. Like most software development projects—and make no mistake, a test automation project is a software development project—we started out with a less-than-optimal design. Maintaining that automation suite over the past few years taught us a few things about improving that design; when working on new test automation projects, I apply those lessons and things go more smoothly. In this article, I’ll show you how our scripts started out, how they ended up, and how we got there.

Starting Out: Our Script Is a List of Steps
Consider the following example test script, written in English. If you are experienced with scripting, imagine that this script is written in your favorite scripting language:

CHECK: Users Can View Their Current Balance
Open your browser
Navigate to www.brittlebank.com
In the field called “Username” enter “bank_test_account_1”
In the field called “password” enter “password1”
Click the button called “Login”
In the dropdown called “Choose a security question:” choose “What is your mother’s middle name?”
In the field called “Answer:” enter “Anne”
Click the button called “Continue”
Click the link called “My balance”
The value “1234.56” should be shown next to the label “Current balance:”

While this is a pretty simple script (for brevity’s sake), it exhibits a serious problem: it’s brittle. An automation suite is said to be brittle if it is hard to maintain. For example, there might be changes in the interface (“Login” becomes “Sign Me In!”), which means you have to fix 500 scripts in 500 places.

Brittleness will kill an automation suite. It’s all over as soon as someone says “We shouldn’t make that change; fixing the scripts will take too much time.” Automation should speed development up, not slow it down.

Luckily, this is a solvable problem. By using the right abstractions, we can minimize rework and fix the brittleness problem.

What Do We Mean By “Abstractions?”
An abstraction hides the details of how work is performed. We use abstractions to make our intentions more clear, to make our code less confusing, and, most importantly, to minimize duplication in our code. Duplication equals brittleness, which equals rework, which equals a dead, abandoned automation suite.

I like to imagine a script as a foreman on a construction job with the abstractions being the foreman’s employees. The foreman doesn’t wire up the house, install drywall, or install toilets. He doesn’t know how to do that stuff and he doesn’t want to know. He has subordinates for that. Let your scripts hire some abstractions so that the script can focus on its job of communicating with you about the work to be done.

Looking back at our script, we can see now that it knows too much and does too much. The script is doing all the work itself. That’s bad, because the next script is going to have to do all the work itself, too. It won’t scale: when things change, every script is going to have to change, and we’ll experience death-by-brittleness.

The term refactoring means “improving the design of our code without changing its behavior.” Let’s start refactoring our script to make it less brittle.

Refactoring Step One: The Pages Learn How to Do Their Own Work
The first thing we notice is that the script knows too much about how to find elements on pages. If, for instance, we want to change the wording on a field, we don’t want all of our tests to break. We can solve this by making “page objects” that know how to find the fields on the page and use them. Our script talks to those and abstracts away the details of finding fields.

CHECK: Users Can View Their Current Balance
Open your browser
Navigate to www.brittlebank.com
On the Login page, log in with username “bank_test_account_1” and password “password1”
On the Security Question page, choose question “What is your mother’s middle name?” and answer “Anne”
On the User Profile page, navigate to My Balance
On the Current Balance page, the displayed balance should be “1234.56”

So, this is a bit nicer and a bit less brittle. Our script knows there’s a login page that uses usernames and passwords, but it trusts the page to do its thing. If that page changes, the definition of the page needs to change, but only in one place, with minimal impact on team velocity. We’re not done, though. There are still a few things that can make this better.

Refactoring Step Two: We Hire Workflow Helpers
There’s still a problem here: what if we get rid of security questions? What if they get moved to the Login page? The presence of a security questions page is presumed by every script, so we’re still too brittle. We can solve this problem by adding an additional layer of abstraction that I call a workflow helper. Think of workflow helpers as subcontractors that the head contractor (the script) hires so that he won’t have to work directly with pages:

CHECK: Users Can View Their Current Balance
Open your browser and go to www.brittlebank.com
Log in with username “bank_test_account_1”, password “password1”, question “What is your mother’s middle name?” and answer “Anne”
The logged-in user’s displayed balance should be “1234.56”

We’ve got three helpers behind the scenes: One that makes browsers navigate to websites, one that logs in users (however that has to happen), and one that goes wherever you go to check balances.

We’re almost there, but we still have two problems. The first problem is that each one of our scripts has to contain all of the data required to log in; the login information might someday change, so we’re still too brittle. In addition, the details of logging in are not pertinent to what’s being checked. As someone reading this script, I don’t care what the password or security question is; I just want to know what this script does.

Refactoring Step Three: We Hide Our Test Data in a Default Data Container

Let’s assume that in most of our scripts, we use the same user. We don’t need or want to put details about that user into every test. So, let’s hide all that information in what I call a default data container:

CHECK: Users Can View Their Current Balance
Open your browser and go to the default banking test site
Log in with the default user
The logged-in user’s displayed balance should be the default user’s current balance

The default user knows its own username, password, question, and answer. The login helper and the balance-checking helper both just ask a user for his information and run with it. You can also hand them different users (i.e.. a user that doesn’t know the answer to the security question) and check the system’s behavior with that data, too.

Easy Mode
Doing these refactorings seems like a lot of work, but the payoff is absolutely worth the cost, in my experience. Using the right abstractions fixes the brittleness problem, with the side benefit of making the scripts much easier to read. In addition, using the right abstractions will make new scripts much easier to create, even for people that don’t have much experience with scripting. This makes the success of your automation project more feasible.