Interviewing? Look for quick thinking, not 'right answers'
Coming up with a business concept is only half the battle. To make your idea come to fruition, you’re going to need a good software engineer. Unfortunately, finding one isn’t an easy process. 
Yesterday, we discussed the six key attributes you should be looking for as you talk with prospective candidates, but getting a read on these in a single discussion isn’t always possible. To ensure you get a holistic view, though, it’s critical to structure your interview correctly.
When I speak with candidates, especially developers, I prefer to have the discussion revolve around an in-depth programming and problem-solving exercise – preferably one that requires the use of a white board. You can use a new question each time, but I prefer to stick with a small number of questions that I have learned well over the years. Over time, it becomes easier to calibrate a good answer if you’ve seen many people attempt it.
I’m not interviewing for the right answer to the questions I ask. Instead, I want to see how the candidate thinks on their feet, and whether they can engage in collaborative problem solving. So I always frame interview questions as if we were solving a real-life problem, even if the rules are a little far-fetched. I’ll then act as their “product manager” who can ask questions of imaginary customers to learn what they think. (I also act as their combined compiler, interactive debugger, and QA tester.)
From this, I can quickly learn a lot from how interested a candidate is in why they are being asked to solve a particular problem. How do they know when they’re done? What kind of solution is good enough? Do they get regular feedback as they go, or do they prefer to think, think, think and then dazzle with the big reveal?
My experience is that candidates who “know” the right answer do substantially worse than candidates who know nothing of the field. That’s because they spend time trying to remember the “correct” solution, instead of working on the problem together.
When they become employees, such candidates have a tendency to tell others that they know the answer when they only suspect that they do. In a real-world situation, they tend to wind up without credibility or are forced to resort to bullying. Either way, they are probably not a good fit for a startup.
A good question, no matter the field, had two key attributes: it has sufficient depth that affords a lot of follow-ups, but also has a first iteration that’s very simple. An amazing number of candidates cannot follow the instruction to ‘do the simplest thing that could possibly work‘ – which is a key tenet of agile development.
Some questions have a natural escalation path and others require some more creativity. In order to get a good read on a candidate, it’s important to be flexible, so you can keep the interview focused on areas where they have to think on their feet, and admit what they don’t know.
The real test for candidates is in the follow-ups. As you probe into their knowledge, can you find gaps or inconsistencies? The goal is to drive the interview to a point where the candidate is contemplating something new.
Eventually, either the candidate will admit they don’t know or will wind up teaching you something new. Either way, they’ve revealed important information. By keeping the questions flexible and constantly probing deeper into what they mean, or why they said a particular thing, you can keep the interview in the all-important zone of ‘not-knowing’ the answer to your question..
There are three degrees of ‘not-knowing’ – and you want to spend as much time as possible in two of those.
1) Doesn’t know, but can figure it out. When you start to probe the edges of someone’s real skills, they will start to say “I don’t know” and then proceed to reason out the answer, if you give them time.
2) Doesn’t know, but can deduce it given the key principles. If you fill a candidate in on the basic rules of a problem, can they utilize those to solve the problem? This has real world implications. It’s quite likely that at some point, someone else will want to teach them something new, but won’t have time to explain all of the details. Can the candidate listen to what’s being said and then put that new knowledge to work?
3) Doesn’t understand the question. Most questions require a surprising amount of context to answer. It doesn’t do you any good to beat someone up by forcing him or her through terrain that’s too far afield from their actual area of expertise. You might decide that this knowledge is critical for the job you’re hiring for, and that’s fine. But it’s disrespectful and inefficient to waste the candidate’s time. Move on.
You want to keep as much of the interview split between the first and second of these degrees as possible. Keep your questions focused on the boundaries of what they know. That’s the only way to probe for agility and brains – and the best way to probe for communication.
In the real world, the vast majority of time (especially in startups) is spent encountering novel situations without a clear answer. What matters is how good a candidate’s thinking is at times like those, and how well they can communicate it.
Next Story: How-to video site 5min scores $7.5M for wider syndication
Previous Story: DeviceVM brings search to its 'instant-on' Splashtop software
People: Jerry Buhlmann
