Behavior-Driven Development and Automation: Establishing Order

My colleague Greg Paskal recently sent a message with the subject line “Behavior-Driven Development - Legit or Fantasy.” In the email, Greg posed several great questions and concerns regarding BDD. Before I had time to get my thoughts together on the matter, my inbox was full of replies. I noticed that much of the conversation was automation-centric. This struck me because I’d recently had other conversations regarding BDD and automation.

Those points helped form my contribution to the conversation, and I’d like to share it with you here.

Putting the Cart before the Horse

From a Scrum point of view, I think of a story as a placeholder for a conversation. If we can agree with that as a good enough definition, then we can see behavior-driven development or acceptance test–driven development as a framework to help teams have that conversation.

The general idea is that during the conversation, the cross-functional Scrum team, including the ScrumMaster, product owner, and the technical staff, gains some sort of shared understanding of what the story is supposed to produce. The BDD specifications (notice I didn't say test cases) become examples: testable artifacts that are the result of the shared understanding. The fact that these specifications are testable lends them to being treated as test cases, for better or for worse. It doesn't help the case much that often these development frameworks are lumped into a category called "test-first approaches."

Now that we have these testable things, if they are written down, can't we parse them and automate everything? Absolutely. Thus, you have BDD tools such as Cucumber, SpecFlow, and Fit/FitNesse.

The biggest flaw I see in all of this is people saying things like, "Oh, yeah, we're doing BDD, so we're going to use Cucumber." The fact that we can specify the conversation results (i.e., the BDD specifications) in a way that can be parsed, and that we have additional software to check the product against the specifications, is thought of as merely interesting—it’s a bonus, a convenience, that we can do that checking via a machine. The result of the conversations is not automation. The result is a shared understanding of the stories; the specifications are the artifacts of that shared understanding.

Here's the dirty little secret: The automation is not the hard part; the changing of your whole cross-functional team to be "test-first" is. Most teams aren't set up for this. Here are some common problems:

• Developers don't want to wait until there are test cases written before writing any code
• Testers are not used to being at the front of the pack, so it takes time for them to acclimate to that role
• Product owners, product managers, and other assorted business people are not as available as they should be, so they can't or don't participate in the conversation as much as is required

This change in mindset is the hard stuff.

If you want to take a BDD approach, do just that: Work on getting the approach right, and forget about the automation. If you can't get the approach and corresponding culture right, BDD will fail, either partially or completely.

For BDD, the conversations are the foundation; we need them in order to even come close to building the correct thing. Once they are in place, then we can talk about the bonus that is automated checking of the specifications.

Frameworks Shaping Script Creation

Yes, you can use BDD-esque tools outside a BDD approach. I used to have a different mindset on this: I thought these tools added an unnecessary layer of complexity if you weren’t doing BDD, unless you had nonprogrammers participating in creating automation script. I've become more moderate on that stance.

Before that email conversation about BDD and automation, I had talked with another colleague, Chelsea Lovas, about her experience creating scripts in different types of frameworks. She used to use the automation framework we created at a previous company that was based on pure script writing. She now uses the framework we have at our current company, which uses SpecFlow as a layer we can leverage to automate BDD specifications. She says she can create scripts far faster using the current framework as opposed to the previous one.

Part of the speed increase is due to VisualStudio’s IntelliSense, as well as a more refined approach to page objects. Nevertheless, Chelsea says SpecFlow gives her some reusability that was missing in our other framework. She also finds that the shared "steps" in SpecFlow have helped reduce maintenance effort. Her team does some BDD-like work, but many of the scripts are created after the code is written. All in all, it's hard to argue with results.

If you take one thing away from my thoughts, I hope it's this: Behavior-driven development has nothing to do with automation; the fact that automation is derivable from BDD specifications is a convenience, not the primary focus.