In this interview with online editor Jonathan Vanian, Bob Galen discusses how agile has influenced testing over the years, the role of a tester in an agile project, and what exactly is “session-based exploratory testing.”
Bob Galen is a leading agile consultant with over twenty-five years of experience working in the fields of software technology and product development. He also authored the book, Software Endgames, and is a speaker at STAREAST where he will be presenting two sessions relating to testing in agile environments. In this interview with online editor Jonathan Vanian, Bob discusses how agile has influenced testing over the years, the role of a tester in an agile project, and what exactly is “session-based exploratory testing.”
Jonathan Vanian: How long have you been working on agile projects?
Bob Galen: I was working at Lucent [Technologies] in 2000 and we were an early adopter of extreme programming. Before that, I was living in Connecticut and I actually stumbled across the original papers about Scrum. They were published for OOPSLA by Schwaber, Beedle, Sutherland—those guys. I was really intrigued by that. I was doing stand-ups right around ’97 or ’95 when they were first describing it. I don’t think I was officially Scrum, but I was really intrigued by the methods. So, maybe for the last ten to twelve years.
JV: Why did you decide to use these methods?
BG: I think I’m aligned with servant leadership. I’ve been a manager for maybe twenty-five years leading software development teams. I remember I was a senior architect or senior engineer, and I came into my office one morning and there was this org. chart on my chair with names on it. My boss and every programmer at the time were introverted, and we didn’t like to communicate with each other. So he sort of like, plopped paper on my chair and was like, “Bob”—he didn’t even say to me, I had to go and ask—“Oh, that’s your team. I just promoted you to be a manager.” I didn’t even know half of the people on the org chart. I had to go find them.
But I’ve been leading for a long time and early on I made those quintessential mistakes of trying to tell people what to do. Very quickly I discovered that the service model and empowering teams were a much more effective way of getting stuff done.
So when I stumbled on agile, it sort of aligned really well with some of my core principles. It was sort of this beautiful convergence with the methods of my style and it seemed to work very well for me. It also gave me a package. What I like about agile is that it sort of gives you a toolbox or package deal for how to communicate a set of things that always work. Like, pair programming always worked. I paired in the late ‘70s and early ‘80s, it just wasn’t [labeled] XP—it wasn’t sexy. We didn’t really think it was a novel thing to do, but agile gave it a name and a package to all the stuff that we all knew.
JV: It’s only recently that people have put a name to these practices that have been around for so long.
BG: Exactly. Doing unit tests first—that’s not novel with agile. Ward Cunningham and a lot of people were talking about that earlier, but they didn’t give it the TDD acronym. They weren’t exactly fixated on unit test frameworks or things like that, but the model, the approach, and the technique were very effective. I remember doing it with assembly code for gosh sakes with very simple text editors and very little in the way of tooling. But we did test first.
JV: As agile has gotten more popular, how have you seen testing change over the years?
BG: I think it’s getting more empowered, the way I think about it. I’m not a tester. I didn’t grow up as a tester, I grew up as a developer, and along the way I started leading test teams. I remember sending myself to testing classes because I felt very uncomfortable leading people when I didn’t know what the heck they were doing. I was reading and studying testing, and I would hang out and test with folks just so I could sort of effectively lead them. One of my observations was that for years and years, testers were sort of woeful second-class citizens. Developers came first. They were always paid more and they were always sort of respected more. That’s a reality, I think, even for today. There wasn’t a lot of technical respect. It wasn’t looked on as a technical profession. People used to joke that testers would just be keyboard monkeys. There were these jokes going around saying that you would just hit keys and randomly ran tests. I never saw testing that way, personally. I didn’t see them as developers, but I didn’t see them as these woeful test monkeys. I saw them as a profession—as something that really saved our butts as development, and it was an incredibly valuable role if done well. And I’ve tried to sort of motivate them. I tried to sort of paint a picture to understand and be empathetic.
I think what agile has done, fast-forwarding, is that it’s sort of leveling that playing field, I hope, where testers are becoming sort of equal citizens in the team—a very valuable role in the team. They are starting to get credit. They are starting to be looked at as being a partner with development. Now this isn’t across the board. It’s still evolving. I run into a lot of agile teams that look at testers and they throw stuff over the wall to them. So [that stigma] is still there, and it’s quite prevalent, but it’s changing. I think that’s one of the most refreshing parts of the agile evolution—what it’s done to traditional testers. They have to work hard, but they can be embraced as effective team members. I don’t think that happened years ago.
JV: Do you think that because some of the methodologies used in agile—for example, the daily stand-ups—involve so much team interaction, that in some way these methodologies have allowed for testers to become more involved or at least recognized more often?
BG: I think both. I think they are being embraced and I think they are being challenged. They are given opportunities. Now it’s a choice for testers. I still try to throw the gauntlet down to testers and say, “It’s not pretty. There is no free ride. You have to step in. You have to work hard. You have to transform yourself to be an agile team member.” If you weren’t comfortable learning or if you didn’t think you had to be competitive technically, you have to be in an agile context. You just have to compete, but now it’s recognized if you show the effort. You get embraced, you get the recognition, and you have business impact. I think the product owners love inquisitive, aggressive, and engaged testers. It’s just a wonderful partnership in agile teams with the business side, the tester, and product owners; there’s a lot of synergy there.
JV: It seems that some of these methodologies require a level of assertiveness on part of the tester. How do you go about making them feel more comfortable?
BG: It’s just finding their niche of value. I’ll use iContact as an example of my most recent company where I was building an agile team, and part of that was evangelizing testing—the test role. We had different types of testers. We had some testers who were developers in sheep’s clothing; we had a few Java developers who wanted to test and they had twenty years of experience. They were incredibly seasoned developers who were great testers. We had some middle of the road testers who had solid technology chops in our systems and environments with SQL and database skills, networking skills, performance skills, but they couldn’t write code. And then we had a set of folks who I would call good functional, incredibly solid professional testers, but they weren’t comfortable writing scripts. So we had a spectrum. It was about finding the niche in what all of those folks could do.
So the functional testers could have an impact by evangelizing and really leading their teams with exploratory testing. The scripters—and I’m not trying to stereotype these folks—found their niche and were supplementing some of the automation code. There are some tools now that actually enabled them to “write automation.” The middle-tier tools like FitNesse and Cucumber allow non-programmers to “automate.” So it really empowered them. For the developers who were testers, they could now start writing user tests and middle-tier tests.
I think the art is empowering them in their niche. Giving them opportunities, but also connecting them to the customer. All of those testers had the responsibility to solve the right problems; To refine the requirements and user stories and to make sure we were getting acceptance. We had them move to the front of the line instead of being at the back of the line having code thrown in their lap and being gatekeepers. They’re sort of still doing a little bit of that, but they’re really trying to move to the front of the line and make sure they’re solving problems. I don’t think there’s anything more cool, from a tester’s point of view, than working with customers. I don’t know if it gets any better than that. I think what motivates them is being apart of that empowerment.
One of the neat things we did at iContact is that we ran very empowered sprint reviews. I don’t know if you’re familiar with the notion of the sprint review, but it’s the ceremony at the end of the sprint where the team gets up and they show off what they did. And we had a whole team do that. Our testers had a stage every two weeks and everyone in the entire team would say, “All right, show me what you’ve done.” We would not just show features, but we would be showing tests and bugs we found, what activity we would be building into automation, and what performance testing we did. It sort of level set all of the activities across the team. I should stop calling them “testers” versus “developers.” It sort of leveled everything and they became contributors.
JV: What do you look for in a strong candidate for a tester in an agile project?
BG: I think I sort of gave you the answer. I don’t think there’s a stereotype. I think it’s someone who has some strength, either strong professional chops or strong exploratory testing chops, or the ability to interface really well. Maybe someone who has some business analysis in their background and understands requirements analysis and then can translate that into a test case. Or a programmer who has gotten tired of programming. I think it’s an elective mix. I think the common ground is just being engaged and not being complacent. Not waiting around and saying, “Here, just give it to me.” Not waiting for 100 percent requirements to be defined and handed to you. Not waiting for your test plant to be 100 percent defined. You need to really get in the game with the customer. You need to be engaged, inquisitive, thick-skinned, and relentless. I think the soft skills are more important to testers in agile environments than the hard skills are, and I think that’s true of the team in general.
Twenty years ago you would have people on a team but they wouldn’t talk. They wouldn’t break bread. I think agile teams truly have to realize that it’s not an individual sport, but it’s a team sport, and you have to play well in that environment, and truly look beyond your domains. I want testers to help developers be inquisitive. Even if you can’t spell PHP—at iContact we were a PHP shop—sit down with the developer and help them write a unit test. Be inquisitive. Show that you have a constructive interest in what they are doing, and expect the same in return because you’re on the same team.
JV: What type of testing techniques are appropriate for agile projects?
BG: There’s a couple core things that I try to amplify. One is this: traditional test automation, which was always approached from the UI down with very monolithic non-developer tools, from HP and Mercury to other tool vendors. It’s not that the tools were bad, but it was this top-down approach. Those approach and strategies were very susceptible to maintenance and challenges from the UI. There’s this muli-tiered approach in agile that’s emerged where you approach some tests from the UI, but very few. You approach some from the middle-tier side, via tools like Cucumber or FitNesse, and you also engage developers to write lots and lots of unit tests in whatever framework you’re using: N unit, X unit, PHP unit—whatever. It’s sort of this multi-tiered approach. I think that’s different, and it’s much more effective. That’s one thing we spend a lot of time on: The disruptive nature and the adjustments you have to make in test automation. It’s a whole team view, not just a testing view of automation.
Another area I’ve gotten great traction in is exploratory testing. I spend a lot of time yakking about exploratory testing in an agile context—exploratory testing and also session-based testing.
JV: Can you briefly explain what is session-based exploratory testing?
BG: You define charters. Let’s say we decompose Skype. You and I are interacting on Skype, so we decompose Skype into a set of charters. We would test the administrative system, the user system, the email system, and the naming system and we would create charters that say “We are going to test this area of the application.” We would decompose the charter and say “We can explore for about ninety minutes in that area.” You might come up with thirty to sixty charters depending on how you decompose any application. The charters are sort of your guardrails that control where you are going to explore and where you are not.
When you are going to attack an application and if we’ve just made changes, the team will decide which charters are relevant for the changes we’ve just made. So they might slice off twenty charters and those charters fit into ninety minutes—a typical time box is ninety minutes. I like to set up pairs. One of the nice benefits that I’ve seen is by doing exploratory testing in pairs you sort of reinforce this teamwork notion in agile. We’ll have developers pair with developers and testers pair with developers. I’ve had Salesforce customer support folks that we pair with testers and developers. It’s an all hands on deck sort of approach where you pair people up, they explore and are guided by the charter, and then you keep notes of your travels. You keep notes on the bumps you find. You keep notes for how to reproduce a bug; you don’t really log bugs.
The focus of an exploratory testing session is to explore as much as you can; to explore the most intriguing areas of your charter and then run them again. You then have a debrief at the end where the team gets together and they talk about what they’ve found. You compare notes and you decide which charters you are going to run next time, not just from what we’ve planned, but from what we’ve found. It’s an incredibly effective way of diving and conquering. It’s very collaborative; it’s very paired.
One of the things we noticed at iContact was that the best testers were actually customer support folks because they had this wonderful view of the customer and their challenges. We had developers and testers fight over who could pair with the customer support agent so they could learn how the app was being used and that would ripple back to the team and into our development activity. So it’s a testing activity, if you think about it, but it has all these side effects where you discover bugs and new test cases. Exploratory testing is a testing activity and an exploration activity. You find juicy bugs, but you also look for ideas for new tests.
