In this interview, Zephyr founder and CEO, Samir Shah, discusses how to make the agile transition as either a big or small company. He also digs into the changing nature of test management, as well as the times when you really need to automate your testing.
Josiah Renaudin: Today, I'm joined by Samir Shah, founder and CEO at Zephyr. Samir, thank you very much for joining us today.
Samir Shah: My pleasure.
Josiah Renaudin: All right. First, could you tell us just a bit about your experience in the industry?
Samir Shah: Josiah, I have been in the software testing space for close to twenty-two years now. I have held many roles along the way, everything from a tester to a CEO of a test management company, and I've been at various Silicon Valley companies—start-ups all the way up to Fortune 50s. I guess I live and breathe testing.
Josiah Renaudin: Plenty of organizations, large and small, are making a move to agile. The agile methodology is really popular right now. What do you see as the biggest hurdles companies need to jump over while making the agile transition?
Samir Shah: The biggest hurdle is actually a mental one. Agile is not a switch that you just throw, that just turns on agile automatically for you. It's a process that you transition into. Getting to the state of agile testing, of being able to test in an agile way—is a journey, the way I look at it. The sooner folks get that, the quicker they will get over that hurdle, which is a mental hurdle of … 'hey, how are you going to get to that point?'
Josiah Renaudin: You talk about it being mental but there are also differences when you are making a transition in a small organization compared to a large organization. Do you think agile is more successful in large or small companies? What causes these specific successes or failures?
Samir Shah: The size of the company is not as important a driver as the actual maturation of the testing organization. More mature process-driven orgs, be that small companies or large ones, immaterial … They can be more successful as they know how to build on any processes, like agile. As long as they have made that mental change, as I was mentioning earlier. Size doesn't matter as much as the maturity of the organization, and their ability to adopt and accept some of the newer ways of doing things.
Josiah Renaudin: You have a really deep knowledge of agile. But you really explore a lot of different areas as well. I want to move on to test management. Can you discuss how important test management has been over time, as well as its current role today?
Samir Shah: Managing the testing process is becoming more and more important these days, right? Let me just share with you a bunch of the factors that make it so. A whole bunch of these factors didn't even exist five or ten years ago. Thinks have dramatically changed and that's why managing the testing process is super important.
Some of the things that have changed [are the] complexity of technologies that are under test. The various test environments that exist today, the plethora of browsers, and devices, and systems, operating systems, etc., the large pool of automation tools. Be it open source, commercial, or home grown, the globally distributed teams. These are no longer co-located. Now it's spread all over the world, in different time zones, etc. Highly integrated and distributed systems under test. These days, you are not building everything yourself. You are integrating with other systems, and other companies' products, etc.
The highly accelerated release cycles that didn't exist a few years ago, growing external dependencies like app stores, and market places, etc., which make you do little things in a different way, and they can manage your testing in a different way, and of course the deeper scrutiny on costs. Without a capable test management solution, teams can really flounder in all of these areas because these things have changed dramatically.
Josiah Renaudin: Over the years, we’ve seen an increase in automation testing. In your mind, how much of the testing process should be automated? How critical can automation be to a healthy testing environment?
Samir Shah: I want to separate automation into the two different categories that it should be in. One, the automation of your actual testing, and the other area is automation of your testing processes. In the first bucket, what can be automated, should not always be automated, right? Some portions of the code, some of the features, could be a throw-away. Some can be complicated cases. ROI has to be measured or calculated before these areas are considered for automation. You just don't go off and automate something just because you can. It might not be worth the trouble.
On the other bucket—what should be automated—a lot of the repetitive manual tasks of running your test department, or your testing cycles. That includes things like planning, assignments, metrics, reporting, and even communicating. The way I look at it, it's 2014, right? We are still sitting with archaic symptoms and archaic ways of managing this process when it should be automated. Let information find you. Automate your test management so you can focus on testing, is how I look at it.
Josiah Renaudin: I like the “let information find you” statement. I think that's really good. Let's move on to QA. Compared to when you started in the industry. You said you have been in for over twenty years. How different does the current QA process look.
Samir Shah: Very different. Very, very different. It went from small teams, that were all co-located, working on a single system under test, very little automation, twelve to eighteen month-long release cycles. Fast forward twenty years later, right? We've got globally distributed teams. We've got massive automation suites, two week release cycles, and quality impact that's measured almost instantaneously as we ship product. It's a whole different ballgame now.
Josiah Renaudin: It's very different from when you started, but would you still encourage companies to shoot for standardized QA within an organization?
Samir Shah: I would to the extent it made sense, right? You got to be practical about it, and not just doing it for the sake of doing it, or to claim that you are standardized. First, determine the value of standardizing certain areas before you go off and do it. For example, shared repositories. That makes sense. You want to reuse your testing assets as much as possible so standardize on that. Standardize on your reporting so that everybody knows what they are looking at irrespective of which group is working on what release, what product, what patch fix, etc.
Automation frameworks, maybe not as much because that can get restrictive based on the tool you are using, or the technology you are testing, and you may not be able to get a standardization across all of your tools. In fact, if you try and attempt that, you might end up with a least common denominator, which might not be good enough.
Things like defect processes, that should be discussed a lot more in detail to see if it makes sense for you to standardize. That's getting to the point, do it to the extent it makes sense, but be very practical about it.
Josiah Renaudin: Not everyone is practical about QA. What companies do you think need to fix their QA. How would you suggest they go about doing that?
Samir Shah: There's a laundry list of companies. I'm not naming names, but the point is, large revenue-backed companies, companies that put out a lot of free software, companies that use antiquated legacy test systems, they need to change. If you are putting out free software, don't let your customers always be your testing team, do put in some amount of testing before you put it out there. Folks that are taking privacy and security lightly, you can't be doing that in this day and age, right? Every ten years, there's a software refresh cycle of new vendors, and new technologies coming of the latest stuff. Adopt those. Let go of the past. These new tools and technologies can only but help in this new world.
Josiah Renaudin: Yeah, and I think letting go of the past, it can be hard, but it's definitely important in an industry that's always advancing. To close, I want to know what you see as the biggest trend moving forward in our industry. We've talked about agile, automation, and QA. What really stands out as the most exciting topic for the future.
Samir Shah: Most exciting. Two things jump into my mind. One, and that's not something new per say, but it's still a trend: quicker, faster releases. That's not going to change. That's here to stay. There's lots happening towards that particular goal, but a ton more can happen there. We can see this over the next few years that people will work very hard toward putting together the right tools, processes, and technologies, to get products out the door faster, and of course with higher quality.
That's one trend, and that's a trend that's been around for awhile, but it continues. The other one, and what some folks thought to be a fad, but it really is a trend, is all things Cloud. Be it management, test management, to test environments, to new cloud architecture that need to be tested. There's a tremendous amount of activity going on there and in the years to come. That I see as one really exciting trend. Mobile is done from a trend perspective. It's here, right? It's established. It's no more a trend, it's part of the standard process.
Josiah Renaudin: Absolutely.
Samir Shah: It's some of these newer areas that still need better tools, better systems, better adaption in the way we go about testing them.
Josiah Renaudin: All right, fantastic. I really appreciate your time, Samir. I appreciate everything you said about different topics. Hopefully, in the future, I can talk to you more about mobile and some other industry trends.
Samir Shah: Absolutely, Josiah, any time.
Samir Shah is the founder and CEO of Zephyr, a venture-backed enterprise software company. Zephyr builds real-time test management software products that empower globally distributed teams to deliver high quality software on time. Previously, he worked as QA manager, QA director, and vice president of other software companies.
