Whether you're organizing a project team or hiring a whole new test group, you need to discover if the testers you're considering can perform the job you need them to perform or not.
When you think about the job, don't just think about the testing—although that's critically important. Consider these skills when you're defining what you need in a tester:
- Selects test techniques that fit the project
- Adapts to the project's conditions (changes testing or test techniques during the project)
- Assesses and reports risks as the project progresses
- Knows when to keep working and when to shift to something else
- Advocates for defects that the tester believes should be fixed before release
- Writes good defect reports
Here are some questions you can ask to determine if the tester candidate is right for your project, especially if the candidate hasn't worked on a project like yours before:
Selecting Test Techniques: "On your most recent project, how did you decide which test techniques to use?" (Wait for the tester to answer you.) Tell me about the project and how you came to your decisions." A follow-up question is, "Have you ever worked on a project where you chose techniques that didn't help you assess the product? What happened?" A final question could be, "What have you learned about test techniques that you apply to your work?"
Adaptability: "Have you ever had to change the course of your testing mid-stream? What did you do? What were the circumstances leading to that change?" (Make sure you wait between each question to full hear what the tester says.) Especially if I'm looking for a more senior tester, I want someone who can tell when the testing isn't discovering what we need to know and can take actions to manage those problems.
Managing Risk: "What risks occurred on your most recent project? Were you able to predict them? What risks did you plan for? When was the last time you were surprised by a risk?" (As you listen to the answers, allow yourself to ask questions in a different order, or modify the questions based on the tester's answers.) Risks happen, and the best way to manage them is to plan for them. If you understand what kinds of risks surprised the tester, you'll know more about that person's blind spots. If a tester hasn't been surprised by a risk in years, I assume the tester hasn't worked on projects critical to the success of his organization.
Changing Tasks: "How did you complete your most recent project? Were you able to perform all the testing you wanted? What tradeoffs did you make and why? Did you have any interim deliverables? What were they?" (Make sure you don't rapid-fire the questions, so the tester has the ability to answer each question as you ask it.) I want to know if the tester considered the information being generated by the tests, and if the tester used any techniques to see if the testing was progressing. I ask these questions to find out if the tester candidate knows when to stick with the current tasks and when to shift work.
Advocating for Defects: "Have you ever wanted developers to fix a defect? When was the most recent time you wanted a defect fixed? What did you do? What eventually happened?" Advocating for defects is not a technical skill, but it can have tremendous impact on the project.
Writing Good Defect Reports: I don't usually question a tester's ability to write good defect reports; I create an audition. Using a product under development or another product with known problems, I ask the tester to take 20-30 minutes to test the product. At the beginning of the audition, I hand the tester a few blank defect reports. As the tester tests, the tester write defect reports as the tester finds problems. At the end of the audition, I review the defect reports with the tester to make sure I understand what the tester meant in the reports. Then I evaluate the defect reports after the tester has moved onto the next interview slot.
In addition, when I'm interviewing someone who hasn't tested a product in our product domain, I'll ask questions about the candidate's ability to learn quickly, such as: "Tell me about a time you joined a project where you didn't know the people or the product. What did you do?" If a candidate hasn't had to learn quickly, you don't know if the candidate will be adroit enough to pick-up necessary skills. If the candidate has been able to learn quickly in a different situation, judge if the situation is similar enough to your project. Then determine if the candidate can adapt to your project.
These behavior-description questions and audition will help you detect if a tester is right for your project—regardless of whether the tester has previously worked on a similar project or not.
If you can't wait for or afford the perfect candidate, look for analogous experience and the ability to climb learning curves. Make sure you ask about specific aspects of previous projects that help you determine how the candidate performed relevant work.
Take the risk out of hiring. Incorporating tests and auditions into your interviewing process will help you build better teams.
