Helpful Tips for Hiring Better Testers

Reflecting on the responsibilities of management in my ten years of hiring, I’ve come to one simple conclusion: My first and most important responsibility is hiring. That’s it. Today I’ll relate the story of how I do that hiring and of my ongoing quest to find the best testers I can.

My interviewing experience began when I was an engineering intern at a small company with less than twenty employees. I was included, but not exactly encouraged, to participate. They were what I would now call “traditional” interviews. The typical questions were “What tech do you know?” “Do you know these tools?” and not much more than that. If someone got along with the interviewer and his answers had been adequate or better, the company would extend an offer.

It was at my next position as a test engineer that I became an involved participant in the interview process. As you might expect, I was woefully unprepared for the job of interviewer and was given no guidance. Falling back on what I knew, I took the traditional route, which included verifying tech and tools experience, determining whether or not the applicant’s answers were acceptable, and gauging if I could be in the same room with the applicant for more then thirty minutes.

This resulted in my first testing-peer on my team and it sucked the life out of me before the first three months were up. He required constant attention and micro-managing or he did nothing. That's when I first woke up with the idea of “hire the best that you can” in my head.

I looked at the good testers around me and tried to identify the “whys” of their success. All of them were driven to learn and capable of adapting to change. If they didn't know a tool or a tech, they learned it. Because under the hood, testing is learning and relearning software everyday. The following are seven changes I made to my interviewing process. 

Change One: Stop Asking Tech and Tool Questions First
It's much more important to find the right mentality. My first interview is now mostly personality based. Followed by, how would you test this, where this’is a generic thing— like a vending machine, a trash bin, or a telephone. I leave it generic enough to encourage the person to start out with questions. The best testers ask lots of questions instead of diving into examples of how they would test, questions like “How much time do I have?”  “What resources do I have?” “What is the goal of this testing?” and on and on and on. Good testers are capable of assuming what kind of this it is and break the problem down into good examples of high-level test design and low-level test specifics. 

Around this point I became the first interviewer of job applicants. I didn’t make the final decision on hiring, but I was the first gateway. It was also around this time that my manager appeared to be under some pressure, and several of my “yes, let’s have further interviews” became orders to “hire this person.” I didn't find this out until several people were hired without any follow-up interviews; my manager soon gave the command to “please fire them now.” 

Change Two: Never Rely on Just Yourself to Interview People
You can be biased and ignore glaring errors that others spot. You might have an off day. Maybe you and this person just mesh and you don't finish your due diligence, consciously or unconsciously.

Regarding my own experience, I moved companies and eventually became the final decision maker. I thought I had interviewing down pat when I got a new challenge: expand the team from six to fifty in about two years. Suddenly, interviewing was the largest part of my job! After the first three months of sluggish hiring I asked HR if I could either be given any or all of the resumes from the people who applied. I needed to conduct a grand experiment and over the course of twelve painful months I discovered a couple of changes.

Change Three: Write a Job Description, Not a Person Description
Testers are a very diverse bunch; what you want someone to accomplish in a job position can be accomplished many different ways. You don't know ahead of time what specific skills that will require, but you do know what goals you want accomplished by this person in three-to-six-to-twelve months (don't you?). I've found some of my best testers from all walks of life: ex-Microsofters, HVAC repairmen, high-school drop-outs, and mechanical engineers. Figure out what the job description should be and don't assume what skills you need to get the job accomplished. (If you’d like more information on job descriptions, I’d refer you to Lou Adler’s work in that area.)

Change Four: If You Have More Questions Do a Follow-up Interview, Not an Offer
In most companies you only get one chance to say yes or no to someone. When someone doesn't work out, it's a long drawn-out emotional process for everyone. So take the time at the beginning and make sure every concern is addressed.

Change Five: If You Still Have to Convince Yourself They are Acceptable After Answering all Your questions, Just Say “No.”  
Listen to your small inner voice. After several interviews and you can't put into words, or at least coherent thoughts, why you still don't want to hire them, you're missing something, consciously or unconsciously. At my current workplace, we keep a standing rule: if they're a maybe, they're a no.

Now, I'm at a company putting together a very small team of experienced testers. This has also required some changes. We are looking for people who already have skills and experience, and can be dropped into projects and succeed with little to no help.

Change Six: Put Them In a Stressful Situation
Testing is stressful; your job is to prove to people how ugly their baby is. My team members are currently testing this out by having interviewees test a small website and find their “most important” bug. Then they get to write it up in two-to-three minutes (try it; it’s pretty hard). We then introduce them to one of our developers, and tell them, "This is the developer that wrote the website. Your next task is to get them to fix your most important bug."

Yes, the developer is real, and he is under instructions to not want to fix anything. There is a fine balance I expect a senior tester to achieve of fighting for his important bug and keeping a long-term relationship with the developer open and professional.

Currently, I'm trying to change up my interviews by selling the company as I am interviewing. However, I'm a horrible salesman. When you are trying to convince senior testers to move to your company you need to be selling the company from first contact to the final offer. 

Change Seven: Retrospect, Reflect, and Improve!
Finally, keep a record of the people you interview and when. Did they get a second interview? Why? Did they make it past the second interview? Why? Did you make them an offer? Did they accept? How did they do after they've been there three-to-four months? Can you make changes to the way you interview to weed out the people who didn't work out in three-to-four months? Or to isolate the people that are doing awesome after three-to-four months? Interviewing is not a static exercise; don't treat it as such. 

After ten years of doing this, those are the lessons I have walked away with. I’m curious, though, what are yours?