When conducting a testing job interview, of course you want to ask questions to be sure the candidate has the skills necessary for the position. But what sorts of questions go too far? Is it ethical to ask a candidate to solve an actual problem your company is experiencing—even if you don't end up hiring him? This article explores some moral gray areas.
In speaking of ethics, I am talking about moral principles accepted by an individual or social group. These concern neither laws nor social conventions, but solely whether something is right or wrong. For me in particular, these are more like nonlegal obligations I try to bind myself to based on my role and the context. I speak for no one but myself and only in situations I have experienced.
I've heard many people complain about job interviews where they were put in front of a whiteboard and coerced into performing complex requests for a group of developers or testers. I experienced one interview where I was pumped for ideas on how to solve a problem and then not hired. It is possible that in these sorts of interviews, the interviewer is actually fishing for ideas to use. While it is not illegal to ask questions related to work, for me, it is unethical to leave a candidate feeling like he was used for the benefit of the organization he is interviewing for. Even if that is not the intent of the interviewer, it will leave a negative impression for the interviewee, which hurts the company's future hiring.
What I want to explore is the ethics of interviewing and how these questions, even when legal, affect an interview.
Go Fish
When I am conducting an interview, I may cite the type of problems my company faces and ask for high-level plans, but I tend to keep the problem space vague. For example, I have asked, “How would you test a search engine like Google?” Search has never been a huge part of the company’s systems, but I have had to test three search engines to some degree thus far in my career. I know how to test such a system (although not at the depth a Google test engineer might) and am not trying to steal ideas from the candidate, but in terms of ethics, is a question like this right?
Am I giving away company secrets? I don’t think so, and my manager knows beforehand roughly how much I will disclose. If I disclose details like source code we use, that might be unethical. From a community guidance perspective, context-driven testing and agile seem to suggest that how much I can disclose depends on the context, and agile in particular chooses to deal with individuals over processes, so I don’t feel it is unethical to pull out a question related to my current work. On the other hand, in a very cagey culture, such as a defense contractor, it might be unethical to explain anything without a signed nondisclosure agreement.
For any question I give to an interviewee, I feel I need to at least have considered the answer first or I might be fishing for ideas. Again using a search engine as an example, I must at least have considered how to test a search system before I ask someone how he would do it. I may use my company's particular search engine as a context because it is in my mind, but I also use at least one other context (e.g., Google). Ethically speaking, I owe it to my employer and coworkers to find the best candidate available. To the candidate I owe a fair comparison to the other candidates.
It May Be the Law, but It's Complicated
This, of course, is about more than just ethics. A debate between Rex Black and Michael Bolton makes it clearer why I'd want to be able to prove my efforts were for a reason, with justification. Not having clear purpose in my questions could open me up to possible litigation. I also want to not get threatened by the company I work for by “leaking secrets,” so I have to make sure I understand the technology and know what is giving away corporate secrets and what isn’t. It is the difference between Google asking some general question involving their ranking system and digging into the exact algorithm used for their ranking system.
When using whiteboards for coding, it is possible to demand too specific a solution. At my current company we have a math function we asked someone to generate that is more complex but still similar to this example. We might ask for someone to create a function that takes two variables in and returns variable1 times variable2 (multiplication). You might have come up with:
function f(var1, var2) {
return var1 * var2;
}
There is nothing wrong with this sort of question. If you need to ask a harder question that requires logic to solve, that too would be ethical.
However, it is not particularly ethical, in my opinion, to ask something that is fundamentally ageist or too biased in knowledge. In most situations you don’t need to require someone to create a new linked list or to cite what browsers Selenium supports or doesn't support. If you need to write a linked list, you’ll Google it, and if you need a list of browsers, guess what? Google is there for you. Someone younger is more likely to have just come from a computer science class and recall a linked list's implementation details better. Now, you can use the question to verify if someone knows how a linked list works in broad degrees, but as soon as you're expecting someone to code one up on a white board, unless your context calls for it, I think you've likely gone too far. Asking about browser support in Selenium fails to tell you whether the candidate simply happens to have a good memory or has deep experience in that area. You are, without follow-up questions, failing to distinguish good candidates from bad candidates, and that, in my opinion, is a failure in your ethical duty to your employer and coworkers. Maybe you just want to hear the “Google it” answer and their thought process, which is fair. Maybe you're using the question to springboard into other subjects. Maybe you are trying to force a candidate to say, "I don't know." These might make sense in your context, but be aware of why you choose these questions.
In our group, we believe that a decent coder will have more difficulty picking up the framework than the technology. I think it would be unethical to exclude candidates because they know a similar but different language. Minor syntax mistakes at a whiteboard would not exclude a candidate in my opinion, but in a different context, such as if you are producing life-and-death devices, I might have different ethical concerns.
How you ultimately interview depends on your context and what matters to you and your organization. However, asking questions without considering what you want to know is a bad idea, at best—and at worst could end up a liability. So, what questions do you ask, and why? What sorts of answers seem reasonable, and when do you have a candidate stop? Also remember that the interviewee will be judging the interviewer on his questions, too, and how the interview is conducted. Have you asked any questions you probably shouldn't, for legal or ethical reasons? Figure these out before your next interview.
User Comments
I had an interview once where they asked me to design a solution to a problem. We ended up designing something that both the interveiwer and I thought was novel enough to be patentable...needless to say I got zero traction on the idea that I should be listed as a co-inventor!
While I am not a lawyer, it sounds like the sort of thing you could go to court over, if it mattered enough to you and assuming you had evidence (notes from the interview, picture of the white board, etc.). As a curiosity, did you ever consider legal action?
Here's the problem with on-the-spot solve-the-problem exercises. They only test for people who can produce an idea for a solution in a terribly short period of time, with NO opportunity to ramp-up on the project or product having the problem. That's not who you are looking for.
If I understand your comment correctly, I'd have to say it really depends on the context. Did the person first gather requirements and an understanding of the needs? That by itself is a helpful hint between someone new to testing and someone more senior. That answers at least one valuable question.
In some cases, you need someone senior enough that they can jump in. If you are Google, interviewing for a Search Test Engineer position, is it really unreasonable to expect someone will know the context of their company? If the person is senior enough, you'd at least expect them to rattle off some test cases about the front page of Google. Many of these things are not even Google-specific problems like Unicode, Language, Login Authentication, checking for 404 pages, disabling javascript, EU cookie warnings (if cookies are created), etc, etc. If a senior engineer can't at least develop hundreds of test cases on such a target rich and well known environment, what are they doing in the interview?
That does not mean you can ask a specific question about your custom No SQL to relational database solution and expect a reasonably deep and useful answer. Is that what you were thinking of when you said a interviewee would need ramp up time?