Naysawn Naderi, a program manager at Microsoft, played an integral role in the R&D process of defining and developing the recently released Microsoft Test Manager 2010. In this interview, an excerpt of which originally appeared in the Sticky ToolLook eNewsletter, Naysawn discusses the requirements for and applications of software testing tools in manual testing.
Naysawn Naderi, a program manager at Microsoft, played an integral role in the R&D process of defining and developing the recently released Microsoft Test Manager 2010. In this interview, he discusses the requirements for and applications of software testing tools in manual testing.
Sticky ToolLook: Many software testing tools focus on the automated side of testing. What are some of the ways testers have used tools like Excel for manual testing, and what are some of the limitations to that approach?
Naysawn Naderi: Our studies do show that Excel, a tool developed in the '90s, continues to be the most popular testing tool in the industry for manual testing. While some manual testers just jot down their test cases on a piece of paper, many prefer Excel, since it can be reused, is digital, and can be shared. However, it provides no customized or optimized testing experience. It amazes me that testers live with it, as it is roughly equivalent to having a developer code using Microsoft Word.
While many use Excel to test, it clearly leaves much to be desired. This includes: taking up a full window while testing, no options for collaborating effectively with the development team, no easy way of sharing what was tested, no ability of sharing steps between test cases, and no linkage to a bug tracking system.
STL: If a majority of testing is still manual rather than automated, do you think there's a market for tools to assist with manual testing in the same way that we've seen a market develop for automation tools?
NN: Absolutely. The studies that we have conducted internally tell us that 70 percent of all testing is performed manually, and the majority of manual testers are testing without any decent tools. Given this lack of tooling, it is no surprise to me that the state of software quality continues to be pretty abysmal.
We believe that Visual Studio Test Professional is the first tool to try to give manual testers a first-class experience for the way that testing is performed today. While I believe we have released something that manual testers really need and that advances the state of the industry, we have really only scratched the surface of the type of tooling which would give manual testers a first-class experience. While we will continue to try to solve some of the problems faced by manual testers going forward, I am sure that others will try to innovate in this space as well.
STL: What are some of the problems that a manual testing tool should solve?
NN: Naturally, it should solve the needs of manual testers, who typically are short of time, work as a part of a team of testers, collaborate extensively with developers, and are passionate about application quality.
To begin, testers need a tool that just allows them to test manually. Specifically, they need a tool which allows them to exploratory test, author manual test cases, run them while focusing on the application under test, fast forward through the uninteresting parts, file rich bugs, share the results of their testing with their team, and validate that bugs are fixed. I am happy to report that Test Professional covers all these needs in a very cool way.
Second, since manual testers and developers work as partners in the development process, each needs an integrated toolset that provides visibility into each other's activities and efficiently allows them to work together. Since Test Professional is built on top of Team Foundation Server, we were able to provide an integrated experience for testers alongside a powerful experience for developers.
For example, in Test Professional when a tester creates a bug, not only are we able to make it extremely rich (complete with video information, a readable log of the actions that took place in the application under test, links to the test case that caused it, and intellitrace logs), but also we are able to present it to developers in an actionable way, where they can immediately jump to the line of code that failed. We feel that if developers are presented with bugs that are irrefutable to reject, they will fix them, and the state of application quality will improve. Further, when the developers fix the bug, they may change some of the lines of code, which will cause some tests that previously passed now to now fail. To check against this, Test Professional tracks the lines of code your tests cover and recommends that testers rerun the test cases that cover the code that has changed from build to build.
Third, manual testers need a tool that will provide them with a workflow of going from a manual test to an automated test to a test which is run as a part of a build process. Happily, this is a gap which Test Professional also fills via introducing the concept of a recording strip to manual testers. In Test Professional, we allow manual testers automatically to capture a recording strip while testing manually. This recording strip can be leveraged to fast forward through the uninteresting portions of the UI if the test is run again manually, but it can also be leveraged by an automation engineer to be converted into a .NET-coded UI test in Visual Studio. Once converted, a tester can add validation, check in code, and have the new automated test run as a part of the build process.
We feel that this solution should help to address many of the issues that have plagued manual testers for decades.
