STAREAST 2015 Interview with Hans Buwalda on the Challenges of Big Testing

Jennifer Bonine: All right, we are back with another virtual interview, and I have Hans with me. Hans, thanks for joining me.

Hans Buwalda: Thanks for having me.

Jennifer Bonine: Glad to have you here today. Now, we've done this before, so I know you, and we've talked about your background. For the folks that out there that don't know you or haven't talked to you before, can you give them all a bit about kind of how you got here and how you ended up in the business?

Hans Buwalda: Well, I got here by airplane.

Jennifer Bonine: (Laughs).

Hans Buwalda: No, I ...

Jennifer Bonine: I think car, maybe.

Hans Buwalda: (Laughs).

Jennifer Bonine: Did you take a car from the airport?

Hans Buwalda: Yeah, it took awhile to get here, to get to that stage.

Jennifer Bonine: (Laughs).

Hans Buwalda: I'm in software, and anything around software since my high school years.

Jennifer Bonine: Yeah.

Hans Buwalda: I've always loved it. More or less, my background is mathematics, but I went into computer science later on, into management conservatory, and then—accidentally, completely unintended—I bumped into testing. (Laughs).

Jennifer Bonine: (Laughs). We've heard that a lot today. The accidental fluke of testing.

Hans Buwalda: Then I started discovering what a very interesting industry that is.

Jennifer Bonine: Yeah.

Hans Buwalda: How much is still there to be discovered.

Jennifer Bonine: Absolutely.

Hans Buwalda: That it is in a continuous development.

Jennifer Bonine: Hm-mm-hmm (affirmative).

Hans Buwalda: Latching on to develop like agile ...

Jennifer Bonine: Yeah.

Hans Buwalda: ... in a very elegant way. So it caught my interest, and it still does.

Jennifer Bonine: Yeah. When you talk about ... Something that you were very involved in pioneering is around keyword testing and keyword automation. For those that have heard that and used it, that was something you were there at the beginning ...

Hans Buwalda: Yeah.

Jennifer Bonine: ... of this phenomenon. Obviously, as you say, things evolve and they change. How do you see the role of keyword testing in automation today?

Hans Buwalda: I would say the keywords are still a very powerful way of separating the technical task of automation with the more conceptual task of ...

Jennifer Bonine: Hm-mm-hmm (affirmative).

Hans Buwalda: ... test design. I think that still counts. Keywords by themselves are not really a solution.

Jennifer Bonine: Right.

Hans Buwalda: They are not really themselves a silver bullet. It takes more, the test design that you build around them. That makes it successful or not.

Jennifer Bonine: Right.

Hans Buwalda: Like I said in one of my talks, some of the worst projects I've ever seen are keyword projects.

Jennifer Bonine: (Laughs).

Hans Buwalda: It is by no means...

Jennifer Bonine: The silver bullet.

Hans Buwalda: ...a guarantee for success.

Jennifer Bonine: Right, you need more than that to be successful.

Hans Buwalda: Exactly. But what I do see is, it fits very well into agile.

Jennifer Bonine: Hm-mm-hmm (affirmative).

Hans Buwalda: I wasn't prepared for that when agile came. Agile is a much more natural way of letting the various parties in a system develop and cooperate.

Jennifer Bonine: Right.

Hans Buwalda: Particularly the developers, and obviously the testers.

Jennifer Bonine: Hm-mm-hmm (affirmative).

Hans Buwalda: The mere fact that the QA, the testers, are in the same team as the developers…

Jennifer Bonine: Right.

Hans Buwalda: … It's wonderful, I couldn't have dreamed of that when I started ...

Jennifer Bonine: Right.

Hans Buwalda: ... in this industry. Than you can also either put your information engineers there which are usually different people.

Jennifer Bonine: Hm-mm-hmm (affirmative).

Hans Buwalda: It's a different kind of interest and skill set. You can also have a team that caters to several things ...

Jennifer Bonine: Yeah.

Hans Buwalda: ...projects. But also domain experts and end users and people like that, and managers, all of those can be much more involved in the process than was ever possible before.

Jennifer Bonine: Right. Yeah.

Hans Buwalda: It's very beneficial for the keywords, too. Another great thing that came along recently is the behavior-driven development, the BDD. We actually had Dan North here on your show.

Jennifer Bonine: Yeah, we were talking about that earlier today.

Hans Buwalda: You can see that back on the video, I think later on, which is also very cool.

Jennifer Bonine: Yeah.

Hans Buwalda: It's a very similar format. Instead of just the keyword, in which we have a sentence …

Jennifer Bonine: Yep.

Hans Buwalda: What I did was I built tools that translates those keywords into these BDDs and vice versa.

Jennifer Bonine: Yeah.

Hans Buwalda: So you can take your BDDs and then translate them in to keywords.

Jennifer Bonine: Yeah.

Hans Buwalda: Which, I discussed that with them this week, which makes them a whole lot more manageable ...

Jennifer Bonine: Right.

Hans Buwalda: ... and a whole lot more reachable for all kinds of testers.

Jennifer Bonine: Right. What I think is interesting—again the folks watching that aren't here—is things like, you see Dan. You're the pioneer with the keyword-driven testing and automation. Dan comes along and says, "Let's do BDD," a take-off of TDD, test-driven development. He says, "That's not working for me. Behavior makes more sense. The people are getting caught up on the word testing." Then, people like you and Dan can collaborate and say, "Well, here's how you pair these things ..."

Hans Buwalda: Yep.

Jennifer Bonine: ...and then continue to evolve.

Hans Buwalda: He told me that in BDD, he's encountering the same problems that I had with keywords—namely, that it is just an instrument; it's not a silver bullet. If you misuse it, which, according to him, most people do, it's not bringing you anywhere.

Jennifer Bonine: Right.

Hans Buwalda: Just to be fair to Dan, he's about a lot more than keywords.

Jennifer Bonine: Oh, absolutely.

Hans Buwalda: I was so impressed with his keynote yesterday; I encourage everybody to see the video.

Jennifer Bonine: Yeah.

Hans Buwalda: He has such a broad range.

Jennifer Bonine: Mm-hmm (affirmative).

Hans Buwalda: He tells about himself, "I am not a tester." Well, he knows so much about testing ...

Jennifer Bonine: He does.

Hans Buwalda: It makes me jealous.

Jennifer Bonine: Yeah.

Hans Buwalda: It's just incredible how much this guy knows.

Jennifer Bonine: Yeah. He actually says he's an interloper because he's actually a developer, but ...

Hans Buwalda: Yeah.

Jennifer Bonine: One other thing that you talk about, if people haven't heard it yet, is action-based and soap opera testing. If people haven't heard those terms yet, can you explain a little bit about what those are?

Hans Buwalda: Yeah. Soap opera testing is a bit of like an odd one out. You could call it a testing technique. It really came to me in a project where I had to deal with a lot of end-users, people who had no testing or technical experience whatsoever. The people who really were ... To help this, working with customers in retirement planning.

Jennifer Bonine: Yeah.

Hans Buwalda: It had nothing to do with automation. I had a lot of those. We needed a lot of very good test cases very quickly. There was not a lot of time.

Jennifer Bonine: Right.

Hans Buwalda: I had a whole room full of them. What I asked them, "Can you come up with anecdotes? Can you come up with story lines?"

Jennifer Bonine: Yep.

Hans Buwalda: Then we will translate using the actions, because that's what action keywords are good at.

Jennifer Bonine: Yep.

Hans Buwalda: Translate it into tests, and then we will take care of making the test. This was still in the Netherlands, by the way.

Jennifer Bonine: Yeah.

Hans Buwalda: What I asked them to do ... Imagine that you are writing a soap opera. A soap opera is a show with multiple episodes.

Jennifer Bonine: Mm-hmm (affirmative).

Hans Buwalda: Basically, each episode is a test.

Jennifer Bonine: Right.

Hans Buwalda: What happens in the soap opera is that it is about real life, but it is exaggerated, and a lot happens.

Jennifer Bonine: Right.

Hans Buwalda: In a half an hour, more happens than in my entire life.

Jennifer Bonine: (Laughs) Right. Exactly.

Hans Buwalda: I hope the same is true for you.

Jennifer Bonine: Yeah.

Hans Buwalda: It's usually not such good stuff that happens.

Jennifer Bonine: Exactly.

Hans Buwalda: It is ideal for testing; that's exactly what you test when the reader wants it to be a bit condensed. It needs to be about real life.

Jennifer Bonine: Yep.

Hans Buwalda: But it also needs to be a little bit exaggerated. What you do with soap opera is you basically place your scenarios in the business. It's not even worst cases, it's really in the business. You make the business story. Then you let story go back to the test.

Jennifer Bonine: Yeah.

Hans Buwalda: On my website … I have a personal website. It's nothing commercial there—it's called Happy Tester.

Jennifer Bonine: Happy Tester.

Hans Buwalda: Of course, I want testers to be happy.

Jennifer Bonine: (Laughs)

Hans Buwalda: On there is a list of my articles. One of those is about the soap opera testing. You will also find bylines there from Lisa Crispin and from Brian Marick.

Jennifer Bonine: Okay.

Hans Buwalda: Brian Marick has a very good one. He has a story in which cars ... A guy is driving a car.

Jennifer Bonine: Yeah.

Hans Buwalda: The car got stolen. The guy gets another car.

Jennifer Bonine: Uh-huh (affirmative).

Hans Buwalda: Then a car bumps into him, which was his original car.

Jennifer Bonine: (Laughs) It circles back.

Hans Buwalda: Exactly. The idea of a story like that, it's not about that story, it's about how it impacts the system and the test. Can the system and the test handle that?

Jennifer Bonine: Exactly.

Hans Buwalda: Does it get into trouble? That's what soap opera testing is. Now, soap opera and keywords are not necessarily connected.

Jennifer Bonine: Right.

Hans Buwalda: The keywords help a lot in quickly expressing that story.

Jennifer Bonine: Right. I love that it's a good way, or an analogy, taking something that people outside of testing maybe understand very well, right?

Hans Buwalda: Yep.

Jennifer Bonine: You say, "It's like a soap opera," and they go, "Oh, I get that because I've seen those."

Hans Buwalda: Yep, exactly.

Jennifer Bonine: "I don't necessarily get testing, and that's kind of overwhelming to me, and I don't understand how to organize it."

Hans Buwalda: Exactly. You don't have to.

Jennifer Bonine: You relate it.

Hans Buwalda: Yeah. It's very popular in the agile world. Lisa Crispin is talking about it in her book.

Jennifer Bonine: Yeah.

Hans Buwalda: That is because it's really what you want in an agile purpose. You want to work with end users. You want to work with domain experts.

Jennifer Bonine: Yep.

Hans Buwalda: You want to do that in a playful, creative manner.

Jennifer Bonine: Yeah.

Hans Buwalda: At the same time, create some very good tests for things.

Jennifer Bonine: Absolutely.

Hans Buwalda: I'm working on a term. Terminology is always difficult, but I have something: Exploratory test development. Instead, exploratory testing is usually, you have your system in the test and you start trying to understand it.

Jennifer Bonine: Yes.

Hans Buwalda: It's kind of a learning experience. That's how we eventually came up with that term.

Jennifer Bonine: Yeah.

Hans Buwalda: The idea is that you work with the system. In my field, that's not necessary.

Jennifer Bonine: Right.

Hans Buwalda: I saw Michael Bolton ...

Jennifer Bonine: Yeah.

Hans Buwalda: ... do an exercise on the flip over with a group, with an audience, and I noticed, "Hey, you're doing exploratory work, here, but you don't have a system in the test, you're doing it on the flip over."

Jennifer Bonine: Right.

Hans Buwalda: You can create automated tests, but you can still exploratory.

Jennifer Bonine: Right.

Hans Buwalda: That does not necessarily have to be interactive. Even if you are interactive, you learn to only test the UI.

Jennifer Bonine: Right.

Hans Buwalda: You don't really go in the business behind the UI. You might miss a lot of books that way.

Jennifer Bonine: Mm-hmm (affirmative).

Hans Buwalda: That is kind of how that all ties together.

Jennifer Bonine: Right.

Hans Buwalda: The focus is always on the testing and on the business. It's never on the automation.

Jennifer Bonine: Right.

Hans Buwalda: The statement I like to make is successful how to make a testing is not as much a technical challenge as it is a test design challenge. That part makes it work.

Jennifer Bonine: Right. Perfect. If people want more information on various topics you talk about, you mentioned that they can go to the website ...

Hans Buwalda: Yeah, Happy Tester.

Jennifer Bonine: ... Happy Tester, and get some of that information because we just barely get to scratch the surface. Other things: You also talk about big testing challenges that probably a lot of folks out there are facing. As the problem gets more complex, we have more complicated systems, we have more of them, we have more environments, we have virtualization, we have outsourcing, we have automation. There's lots of challenges people are facing as testing gets bigger.

Hans Buwalda: Yep.

Jennifer Bonine: You give a whole session on this. We said we don't have seven hours; you'd have to hunker down and get ready.

Hans Buwalda: I will try to explain it in six and a half hours.

Jennifer Bonine: (laughs) Okay.

Hans Buwalda: That's good enough.

Jennifer Bonine: Six and a half, just get your popcorn ready.

Hans Buwalda: I'm flexible today.

Jennifer Bonine: Perfect. What would you say, where can people get more information on your thoughts on those things if they want to learn more?

Hans Buwalda: Well, I've written and blogged about it. On Tech World, there are a lot of blogs.

Jennifer Bonine: Yes.

Hans Buwalda: Specifically, they ask speakers like myself ...

Jennifer Bonine: Yep.

Hans Buwalda: ... at the visitor conference coming up, can you write a couple of blogs?

Jennifer Bonine: Yep.

Hans Buwalda: I went ahead and, of course the links are on Happy Tester, to talk about it.

Jennifer Bonine: Perfect.

Hans Buwalda: Big testing for me is if the size of the test in itself needs to be managed.

Jennifer Bonine: Yep.

Hans Buwalda: Normally, when you manage tests, you talk about the test ...

Jennifer Bonine: Right.

Hans Buwalda: ... and you talk about the automation, but when the size becomes so big that you need to manage it. I think in Texas they say, "Everything is bigger than everything else."

Jennifer Bonine: (laughs) Exactly.

Hans Buwalda: That is what you mean with big testing.

Jennifer Bonine: Right.

Hans Buwalda: A lot of, especially corporations ...

Jennifer Bonine: Mm-hmm (affirmative).

Hans Buwalda: ... have this problem. They not only have a very large amount of complex systems in the test ...

Jennifer Bonine: Right.

Hans Buwalda: ... but they have lots platforms, and nowadays also the mobile coming up. New platforms come up, but the old ones don't go away.

Jennifer Bonine: Right.

Hans Buwalda: It gets bigger and bigger and more complex, that's worth a tutorial.

Jennifer Bonine: Yeah, absolutely. You should come back and attend.

Hans Buwalda: Okay, that was six and a half hours.

Jennifer Bonine: I know. Good job. You all made it. Thank you so much. We're already out of time. It goes so fast. Again, Happy Tester is one way to find you. Any other ways you want them to know if they have questions or want to contact you?

Hans Buwalda: I work for LogiGear, at the company.

Jennifer Bonine: Yeah.

Hans Buwalda: By the way, we have the coolest test product that you want.

Jennifer Bonine: They can find you there.

Hans Buwalda: Yeah. You can find me there as well.

Jennifer Bonine: Perfect.

Hans Buwalda: Okay. Thanks very much.

Jennifer Bonine: Thanks, Hans. I appreciate it. Thank you.

Hans Buwalda: Yep. It was a pleasure again.

Hans Buwalda has been working with information technology since his high school years. In his career, Hans has gained experience as a developer, manager, and principal consultant for companies and organizations worldwide. He was a pioneer of the keyword approach to testing and automation, now widely used throughout the industry. His approaches to testing, like Action Based Testing and Soap Opera Testing, have helped a variety of customers achieve scalable and maintainable solutions for large and complex testing challenges. Hans is a frequent speaker at STAR conferences and is lead author of Integrated Test Design and Automation.