Josiah Renaudin: Welcome back to another TechWell interview. Today I’m joined by Michael Nauman, a tester at Autodesk and a speaker at this year’s STARWEST Conference. His session will cover how shifting your testing left goes well beyond agile. Michael, thank you for joining us. First, could you tell us a bit about your experience in the industry?
Michael Nauman: I found my way into the testing world through the back door, as a user. I was doing computer-aided design using Autodesk’s AutoCAD for the county of Sacramento, California, for ten years. The opportunity presented itself to work on the AutoCAD test development team, and I have been testing various forms of AutoCAD for twenty years.
I started out as a black box tester, did some test automation, moved back to black box, then to a test lead, then test manager. This was followed by a short stint as a product owner, and I recently moved back to the testing lead for AutoCAD Web.
Josiah Renaudin: You mention that the shift-left concept isn’t exactly new, but it feels like it’s being discussed more than ever these days. Is that mainly because of the emergence of automation and agile practices, or something else?
Michael Nauman: I think it is the ever-evolving nature of the tech industry and the rise of cloud and mobile. The software development lifecycle is measured in hours or minutes these days as compared to months or years a decade ago. For dotcom 1.0, the tools were rudimentary at best. Now we have open source tools and frameworks that solve a lot of the problems of getting a web or mobile application up and running. As the tools evolved, so did the development process. Test-driven development and Extreme Programming seek to increase the efficiency of engineering teams, while the lean startup does the same for product teams. Shifting testing left only makes sense with the natural selection that exists in the tech industry today.
Josiah Renaudin: How important is it to align your DevOps with your test automation, and what would you say is the most general, successful strategy for keeping these in sync?
Michael Nauman: Test automation, as well as infrastructure automation, should never be an afterthought. No code should be submitted without unit and/or integration tests. Test automation is your safety net and you don’t want bugs slipping through it. Once a CI system is up and running it tends to stabilize, at least it should. Test automation, on the other hand, should constantly be updated as functionality is modified or added to your application. Test automation is the critical component of your CI system and it must be trustworthy. Knowing what your test automation covers is just as important as knowing what it doesn’t cover. Missing tests or unreliable tests should be considered debt and therefore tracked, prioritized, and implemented along with feature work.
Josiah Renaudin: Can you dig into the concept of separation of concerns with regards to quality that you’ll be covering during your session?
Michael Nauman: Having clear ownership is important to set responsibilities and accountability for your team. Quality is no different. My team has separated the ownership of the quality of the code versus the quality of the product. The quality of the code concerns the tool chain, architecture, submission process, approach to testing, test automation, build system, etc. Quality of the product centers on the needs of the customer. Does your product solve a problem for a specific customer? Can your customer complete their task using your application? Do customers love your product? You can have a high-quality code base that solves no customer problem. Likewise, having a valuable product from a customer standpoint that is brittle, buggy, or unstable is just as bad. Building a high-quality product is just as important as building the right product for your customers.
Josiah Renaudin: Going “beyond agile” is something you key in on. Is that an admission that agile, by itself, isn’t enough to make a testing team successful? Is agile more of a starting point than anything else?
Michael Nauman: By “agile,” I mean agile Scrum or Scrum. Agile Scrum is a good place to get to and stabilize on if your team is not there yet. If you have multiple dispersed teams, Scrum is a good method to align and coordinate between them. However, after reading The Lean Startup by Eric Ries, I find Scrum a bit confining. Being lean and as efficient as possible gives your team flexibility to react to changing conditions in the market.
Josiah Renaudin: In your mind, is there a perfect balance between what percentage of tests you automate and what percentage you handle manually? Or does it really just depend on the type of project and makeup of the team?
Michael Nauman: It depends on the project and its level of maturity. If you are in the early days of a startup, then a large investment in infrastructure and test automation doesn’t make sense. That said, having basic DevOps is mandatory for builds and a framework for eventual automated tests. Initially, manual testing of an application may be enough. However as an app gains traction, market share, and increased functionality, there will be a tipping point where an investment will need to be made in test automation. Being efficient and lean is critical but also acknowledging any debt that gets accrued. If your team chooses to defer test automation in an early stage, document in your backlog those tasks you are consciously choosing to postpone until later.
Josiah Renaudin: A lot of people hear about shifting left and assume it’ll slow things down since they’re testing earlier and more often in the process. From your experience, how can shifting left actually increase velocity all while guaranteeing better quality?
Michael Nauman: The point of shifting testing left is to reduce the amount of manual testing effort, increase the quality of the builds coming out of your CI system, and reduce the cycle time from your engineer’s machine to production. The later you find defects in the software development lifecycle, the more expensive and risky they are to fix. By shifting testing left, the defects are found either in the design process or on the engineer’s computer, which aligns with test-driven development. Testing early and often gives you confidence by reducing risk and uncertainty as your project scales in terms of functionality and consumers.
Josiah Renaudin: More than anything, what central message do you want to leave with those who attend your session?
Michael Nauman: I’ve seen a lot of things in my two decades doing software testing. Embracing change and new technology is critical to any tester’s career. Continual learning is essential to anyone in the tech industry, and as testing professionals, we should embrace it.
Michael Nauman started out his career doing computer aided design but after a decade made the leap from an AutoCAD user to a tester on the test development team for Autodesk Inc.’s flagship product AutoCAD. He has spent the past twenty years testing various versions of AutoCAD. Michael started out as a black box tester, dabbled in test automation, evolved into a testing lead, managed testers as a QA manager, and now is the product owner of AutoCAD Web. Michael is passionate about AutoCAD, DevOps, bicycles, and surfing.
