The Fundamentals of Agile: An Interview with Jeff Payne

Josiah Renaudin: Today I’m joined by Jeff Payne, the CEO and founder of Coveros and instructor for SQE Training's Fundamentals of Agile Certification class. We'll be speaking on both agile as a whole as well as the different aspects of the course itself. Jeff, thank you very much for joining us today.

Jeff Payne: Thank you, Josiah, for having me.

Josiah Renaudin: Absolutely. Before we start really digging into the material of the course, can you tell us a bit about your background and your experience with the agile methodology?

Jeff Payne: Sure. I’ve been building and testing for twenty-five years. I got first interested in agile really when Windows XP and some of the early agile writing started appearing out into the world and books and papers started being written about agile. What attracted me to it was a couple of things. First of all, as a developer early in my career before I started my first company, when I read about agile, it just made perfect sense to me. It seemed like the types of things that I would want to do and would think would work as a developer.

Second is that running a company at that time that was around quality assurance and testing, I loved the fact that quality was so integrated into agile, because we were always pushing to get people to move quality earlier in the process, do unit testing at the developer level, and to automate when it made sense. Your QA and test and agile just fit perfectly with that. When I decided to start a new company in 2008, I decided to center it around agile and DevOps, which is naturally very closely aligned with agile.

Josiah Renaudin: Yeah, good for you, because of the fact that both of those concepts are really blowing up more and more as we dig deeper and deeper into them. What do you think makes agile so different from traditional methodologies?

Jeff Payne: There’s lots of differences between agile and traditional methodologies. I think most of these are outlined pretty well in the principles of agile that are part of the manifesto. The manifesto is really the defining document for the agile movement. A couple that I like the most, the principles that I think really highlight the differences are, first and foremost, working code is the measure of progress. We always talk about all the different artifacts that we can produce during the software development process, things like requirements, docs, design docs, and obviously the code, and test cases, and test plans, and things like that, and they’re all good things and we need them all, and to review them is a good thing to do.

However, the reality is there's only really one thing that ships to the customer, and that’s compiled code. All the other artifacts that we produce are things that are supposed to help us get to where we want to get to. Agile says, "It’s great to have those things, use those things as you need them, but don’t try to measure your progress based on them." I’ve sat in on a lot of review meetings, and document review meetings, and project review meetings, and you’re always trying to figure out are we on schedule, are we on budget, are we going to get what we want to market? The reality is it’s really difficult to do that if you’re not measuring what you’re actually delivering and that’s working code.

In traditional methodologies, you don’t have working code until really late in the process, so that makes it really hard to use working code as a measure of progress. That’s where the agile approach where you’re building code in small increments and small bite-size chunks is just radically different than traditional methodologies and something I think makes agile work.

The second one that I would highlight is the principle of agile around embracing change. That kind of sounds like motherhood and apple pie, because the world’s changing and we need to change with it, yada, yada, yada, but you'll notice that a lot of traditional methodologies around software are all about trying to control change. There's change review boards, and there's change control, and there’s manage to change this, control change that, but the reality is we don’t control change. Agile says, "We're not been going to control change. Instead of controlling it, we should embrace it, we should expect it to happen and when it happens we should figure out how to deal with it."

The only way you can do that effectively and in a cost-effective way is if you’re not investing in a lot of upfront planning before you build stuff. You really only want to plan—I call it "just-in-time planning"—right before you’re ready to build and test something, because otherwise the world might change out from under you and you'll have spent a lot of time and effort building something or planning out something but then now all that work is rework.

Traditional methodologies we spend a lot of time up front, requirements, architectures, specification, design before we ever build anything, and the world changes while we're doing all that, and sometimes all that work is for naught. So agile says, "Don't do that. Don’t really invest in a feature and planning it out until you’re sure you need it and it's a high enough priority."

Those are two of the ones that I think really highlight the difference between agile and traditional methods.

Josiah Renaudin: You use the word change quite a few times. As you know, people are afraid of change, and especially businesses are afraid of change, especially when there’s money on the line. What sort of tangible improvements can people expect to see after moving away from the current methodology that they’re using, and you can point to these changes and say, "It’s worth making the move?"

Jeff Payne: There's a lot of studies out there that show some of the business benefits of agile. They document and define better quality, decreases in costs, faster time to market, all that stuff. What you don’t hear a lot about are really the team benefits, the underlying things that make those business benefits possible. Those are the ones I think that I'd like to highlight, because without benefits to the software team itself you're not going to really see any of the business benefits.

Some of the biggest team benefits I’ve observed that result in business benefits such as productivity and higher quality are, for instance, increased visibility. The ability to know all during the software development process how much progress is being made that relates right back to this idea of working code as your progress, but also the fact that we’re always communicating and we're keeping the customer in the loop so that everybody knows where we're at in the process. I find that helps a lot because then when there are issues, maybe the customer's changed their minds, we know customers often don’t know what they want until they see it. We're able to show them something earlier and communicate and talk to them about it. They can give us feedback on whether that’s really what they want or not. That can help drive productivity and better quality.

Second is, I find increased team morale when agile's put in place. Teams feel more empowered, they feel like they’re more in control of making decisions, and they have the authority to just get the job done without a lot of distractions from other places. That often drives higher productivity. It certainly drives better morale and people wanting to stick around longer and not so much context switching around the team members.

Last, but not least, I think everyone feels more ownership for quality in an agile team, because if you do it right … Agile teams are evaluated by whether the team delivers or not in your iterations. Not whether a particular person delivers. It’s either the team wins or the team doesn’t. When you get that mentality in place, everybody’s rowing in the same direction and everyone’s helping with quality, which is a welcome relief, quite honestly, from the developer versus tester mentality.

Josiah Renaudin: Absolutely. Even if people do see and hear about these benefits, if it requires a great deal of work, and maybe a complete reworking of the team; that can also be a little bit intimidating. Is adopting agile a major, multi-step extended process? Going through a two-day course will obviously improve your understanding of the methodology, but how long does it actually take to transform a standard team into an agile team?

Jeff Payne: That's a great question. Agile transformation, it is change, and it’s real change, and it’s the kind of change that does take time. It’s not going to happen overnight. Like any change in the organization, it will take time, and it’s hard.

In our experience at Coveros, we’ve done this for many customers over the years, helping them move from a traditional software development process to an agile process. It’s a multi-year process typically for most organizations as a whole to transition to agile. You can get your individual teams typically within an agile process, I’d say within six months.

Typically, a team is to the point where they can successfully implement agile and do it on their own. They might need some continued coaching or support in areas where they’re not strong or they have a deficit. For instance, maybe they in the past haven’t done a lot of test automation and they realize that you need to do some of that, and they’re still looking for the right person to help there, and someone from Coveros or somewhere else is kind of plugging that hole for some period of time, but usually by six months a team is starting to roll.

In order to get to get it to work across the entire organization, you're typically looking at a couple of years. One of the things we always make sure senior management understands is that they’re going to have to stand behind this time frame and this commitment and see it all the way through. There’s going to be bumps in the road. They’ll be ups and downs. Change is hard, but you can’t get part way in and just bail. You really do need to decide, as you mentioned, Josiah, is this really something you want to do and are you committed to spending the time and effort to make it happen? If you do, then you’re very likely going to be very successful and happy with the result.

Josiah Renaudin: A similar topic: Let’s say you do want to go agile but you’re not sure if you want to go 100 percent fully agile, because you're still a little bit concerned about making the full transition. Do you believe that methodologies can be blended? For example, if someone is looking to transition to agile can they also maintain other aspects of waterfall or others semi-similar methodologies?

Jeff Payne: My experience in this says we’ve seen obviously a lot of different ways that people have gotten to agile from where they are, so I would never say that you can’t do anything, but I personally am not a fan of trying to take a hybrid approach to moving towards agile. Mainly because, and this is something we concentrate heavily in the course on, is the importance of the principles of agile. The agile principles have stood the test of time. They lay out the philosophy for how to be agile, and if you’re only doing half of them because you’re still kind of have one foot in your old process, you’re not going to see the benefits and may actually not see any of the benefits.

A perfect example; we often get called in by organizations where they’ve decided to put Scrum in place. By Scrum they're really talking about the ceremonies, daily stand-ups, kickoff, perspective, etc. etc., but down in the trenches the developers and testers aren't doing anything different. There’s still building code, throwing it over the wall, testing code, throwing it back over the wall, fixing code, throwing it back over the wall, retesting code. Now they’re trying to do that really, really, really fast. Now you’ve got to produce working code within a couple of weeks or so.

Our experience is that doesn’t work. We call that, "Scrum or fall." It’s one of the root causes of agile failure that I talk about in my Root Causes tutorial, because you still have one foot in the waterfall land and one foot in agile land. Our advice is that instead of trying to do a hybrid, take one project. or one piece of the organization, one group and focus on getting agile established there. Transform them completely and make sure they’re following the agile principles. Once you’ve done that you can take that as an exemplar group, or exemplar project and start to evangelize and push out to maybe some of the projects that are little bit more difficult to move to agile with.

The last point I'll make is that—I always tell people this in the course—is, "Don't throw the baby out with the bathwater." The first thing we always ask customers is, "Why are you moving to agile?" If their answer is, "Oh, it's the newest cool thing," or, "We’ve heard it’s really great," or whatever, my follow-up question is, "What’s not working in your current process?" They say, "We have happy customers. We deliver software fast enough for them. They don’t complain. We have good quality. Our people are happy." My response would be, "Why are you moving to agile, then? It sounds like everything is working, so what are you going to get with agile other than two years of turmoil because it is going to take time?"

Be careful, and don't move to agile if everything is working. You should use agile as a way to improve, and if you really feel like there’s some places that your organization is not delivering what's needed to the market, then you consider agile. If you’re not having those problems, don’t move to something new. Change is hard.

Josiah Renaudin: Yeah, change is hard. Of course, along the way while you’re making that change, people are going to have missteps. They’re going to do things that might actually prevent their agile transition. What are some of the most common mistakes that people run into when trying to become agile that hopefully you’ll be able to help them avoid, or at least recognize, with your class?

Jeff Payne: Yes. The first one is what I call "doing agile versus being agile." This course is all about how to be agile. A lot of times we see organizations struggle when they try to do agile. That means they have a checklist as to the things that you’re supposed to be doing and they walk around and make sure people are doing what they’re supposed to be doing, but the people are vested in and thinking about agile based on the principles. They’re not bought into whole team quality and doing the things that are necessary to drive productivity and improve quality. They’re just following a checklist. If you try to do agile, it won’t work.

We see this a lot with ScrumMasters. They’ve got ScrumMaster training, they're not a vetted certified ScrumMaster, but unfortunately they haven’t learned how to be agile. All they’ve learned is, "Here’s your clipboard sort of things you need to walk around and do." That’s not going to work.

That’s the first thing we try to get people to understand. The way we do that in the course is simple. We focus a lot of what we’re talking about on the principles and how the principles relate to practices, because if you can get them related then you can always use the principles as your guiding light. If you start to see it moving away from the principles of agile, you're starting to do agile instead of be agile—you can course correct and adjust. That’s the first one.

The second one I mentioned when I talked about struggles with developers and testers; we still see a lot of organizations that haven’t figured out how to get their developers and their testers to work together. Not only on an hour-by-hour basis but even on a week-by-week basis, they can’t figure out how to get them to work together. These are two groups that haven't traditionally spent a lot of time together. They're in separate groups in a traditional software development process and they don’t know how to work together. What are they going to do if they sit together? How are they going to work? If you don’t tackle that you’re going to struggle in agile because you're going too many hand offs. Again, the whole idea behind agile is to get rid of all these hand offs that we have in our process and get everybody working together to build quality into software instead of trying to test, fix, retest, fix, retest, fix.

The third one I’ll mention is that the team isn't maniacally focused on improvement. You're going to start with a process in agile, and I tell people in our class I don’t really care which one you start with, you can start with whatever you want. Make sure it meets the principles of agile, but start wherever you want. I’m not vested or tied to one particular approach within agile, because the reality is you're going to find out some of what that approach provides you works well, and some of it just doesn’t fit. Maybe it’s your culture, maybe it’s your people, maybe it's your market, maybe it’s your tools, whatever it is something is not going to work, and you’re going to change. But if you never take that improvement step and continually improve your process then you’re going to be stuck where you started, and usually where you started isn't ideal for your organization.

One last one, which is important, is I mentioned DevOps earlier, this whole concept of automating your build, your test, and your deployment as much as is possible in your environment is critical in agile, because you’re going to have to do things pretty quickly, and if you don’t figure out how to automate some of the pieces of that puzzle you’re going to find you don’t have time to get done everything you need to get done, particular as your software starts to grow in size.

Josiah Renaudin: Your course covers the fundamentals of agile, but let’s say someone is more well-versed in the methodology, but still wants to dig a little bit deeper will that be something they could do? Do you encourage people who even are already agile and do have a pretty good idea of what agile is do you encourage those people also attend?

Jeff Payne: We get all sorts of people in our class. I always ask up front when I teach the class, "What’s your background or experience in agile?" You'll get everything from, "My boss came to me yesterday and told me I was going to this course. I just learned how to spell it." Two people who said, "I’ve been doing agile for a few years. It seems to be working okay, but I want to take a step back and hear a different perspective and see if we’re not doing things right, because I’m not seeing everything that I expected to see."

This course will work for all those people. We're told again and again by students who have taken other agile classes and are doing agile that the Fundamentals of Agile course we have at Coveros is by far the best introductory agile course they’ve ever taken.

The reality is, a lot of the agile training that is out there is very prescriptive—meaning it’s focused on a particular methodology that teaches Scrum, or it teaches lean, or it teaches kanban, or it teaches a particular set of practices. That’s all great, but again, that’s kind of focusing on doing agile. Our course really highlights and focuses on the principles of agile and then tries to relate different approaches to it. We will cover Scrum, and kanban, and lean, and Extreme Programming, and test-driven development, and things like that, but we always try to tie it back to the principles, so that irrespective of which approach you decide to take, you’re grounded in how you can improve the process and how to be agile instead of just do agile.

The second is that the course is very practical. That’s because at Coveros we build software using agile for a living. That’s what we do. We're practitioners. We're pragmatic. We're practitioners and we do this for a living. What we’re teaching isn't what the book says you should do, it's what we found works. We have a lot of knowledge and information about how to deal with difficult situations or unique situations that the books don’t really cover.

I’m a big advocate that if you’re going to go through an agile transformation, and you’re going to build out essentially your own agile process over time as you improve, that you really should do that in the context of working with people who are pragmatic about it, and aren't just giving you book knowledge. That doesn’t really take you very far. It’s very pragmatic and that’s why people like it.

Josiah Renaudin: Absolutely. Like you said, the course is for people of all different ranges. It just depends on who shows up and what they’re looking for. What can attendees expect to earn upon the class's completion?

Jeff Payne: Yes. First and foremost, they earn a good understanding of agile, I would say. Also we do give out a number of things that are helpful for people as they’re advancing their career. The course is certified by both the IC Agile Consortium, which is a Consortium of Agile professionals that gives out agile certifications for various roles within agile, and also the PMI, which gives out project management certifications, and accreditations, as well as agile accreditations.

Our course, if you go through it, you get two things. You get the first level designation from IC Agile, called the Certified Agile Professional; it signifies that you are a certified agile professional after taking this course. Then there are a number of courses you can take thereafter to get more role-specific certifications, so you become a certified agile developer, a certified agile tester, certified agile coach, whatever your path is. We also give fifteen PDUs that can be used to maintain either your existing PMP from PMI Institute, or your agile certified professional certificates that you receive from PMI as well.

Josiah Renaudin: Fantastic. I really appreciate you talking to me today, Jeff. I just wanted to close on a broad question. Do you see agile having staying power? I’m assuming I'm going to get a yes, because you’re pretty invested in it. It’s very popular and successful so far, but do you really think it’s here to stay? Is it worth investing into and to make that, like you say, that two year difficult transition to go agile?

Jeff Payne: I think it is, absolutely, and not just because I run an agile company. That’s why I founded it, because I did believe it had legs. As a business executive, regardless of your business ,you’re always trying to figure out whether anything new is, I always say, a trend or a fad. Is this something that’s going to be around for a while that I can bank on, invest in, or is this a fad that’s popped out and is going to quickly go away, and it would be a waste a time and effort to look at? I think agile’s a trend. It’s really because it’s not a specific methodology or set of practices. It’s because it is a set of principles.

The principles are spot on. As mentioned, when I saw the principles and read about agile, I said that’s exactly the way we should build software. Those are exactly the things I believe. There’s a lot of people out there that look at the principles and the manifesto and say, "Yes, that is how you build good software."

I think because it's a set of principles and the methods and practices and tools underneath it can morph and grow and change that it has some legs. It'll grow up. It's still new. It's only ten, fifteen years old, but you’ll see unique and innovative concepts come out of it. I would argue DevOps came out of it. The idea of starting to embrace and bring operations into an agile process, which DevOps is all about all centered around this concept in agile of getting everybody in and communicating so that we don’t make mistakes, so we find problems earlier, and that we're automating as much as we can.

I believe things like DevOps will continue to pop out of agile and become their own thing but related to agile. Agile should get some credit for driving that entire philosophy around getting everybody working together to make software more successful. I do think it’ll be around for a long time.

Josiah Renaudin: All right, that's good to hear. That’s encouraging for a lot of people. Thank you very much for speaking with me again today, Jeff. I’m looking forward to talking to you in the future more about agile, DevOps, and a medley of other topics.

Jeff Payne: Thank you very much. I enjoyed it.

Jeffery Payne is CEO and founder of Coveros, Inc., where he has led the startup and growth of the company. Prior to Coveros, Jeffery was Chairman of the Board, CEO, and co-founder of Cigital, Inc. Under his direction, Cigital became a leader in software security and software quality solutions, helping clients mitigate the business risks associated with failed software. Jeffery is a recognized software expert and speaks to companies nationwide about the business risks of software failure. He has been a keynote and featured speaker at business technology conferences and frequently testifies before Congress on issues of national importance, including intellectual property rights, cyber terrorism, and software quality.