How do you hire a great engineering manager?

It’s a question I’ve been grappling with a lot, as the company I’ve been leading has doubled in size every year for the past five years. At times, I’ve thought great engineering managers might not even exist. What is it that a great engineering manager has? Obviously, technical ability is key, but the purpose of this job isn’t just to hack technical problems. It’s to hack the very different human and business problems. There are certain attributes they need, such as the ability to be critically introspective, to be able to drive to root causes, challenging underlying assumptions, and obviously, they need great interpersonal skills.

But is that it? If I could test for all these attributes, and a candidate had them, would I have a great leader in front of me?

We learned the hard way that the answer to that question is actually no. Testing for all the various attributes of a great candidate in isolation isn’t enough. What matters even more is the way in which all these attributes interact together. That’s what makes for a truly great engineering leader.

So, as the final step in our selection process, we sought a way to see if a candidate’s whole was greater than the sum of their parts. This is the test we came up with:

It’s been a year since you joined Medallia as an engineering leader. You were chartered to lead a group of our finest engineers. Please prepare a retrospective presentation with the engineering leadership team explaining:

  • What did you do during the first six weeks on the job?
  • How did you manage your engineers?
  • How did you lead them?
  • What have you accomplished during this first year?
  • What has your team accomplished?
  • How did you and your team evolve?
  • Please provide examples illustrating best practices that can be shared with others in your position.

As part of forming answers to these questions, we give candidates the opportunity to join the engineering team’s daily stand-up meetings and then hold one-on-one meetings with team members of their choosing. Afterward, we give them 48 hours or so to let those experiences percolate before coming back in for their presentation. The results of this challenge don’t just provide a depth of insight into candidates that was previously unavailable; they leave us with 100 percent certainty of whether we are hiring the right person or not.

How is that possible? Think about what it shows us. Right from the get-go, as they spend real time with the team and watch us work, we get to see who the candidate chooses to interact with and the person does so. We can gauge if the person has the right instincts around the team’s resources and dynamics, if he or she is able to envision how to positively leverage those resources. Even the language the person uses can be very telling of personal philosophy and world view.

For example, subtle word choices can reveal the difference between someone who is all about empowering others and someone who is all about power. And given that candidates give their presentations in a room filled with senior leaders at Medallia, how they react to feedback in this environment is a great reveal, not just of their dynamism in the face of questions, criticism, and pushback, but specifically how well their skills will fit into the context of our engineering team.

While some tech companies require you to sign an non-disclosure agreement before taking their engineering challenges, I’m sharing ours with everyone. Simply put, it’s a process that is respectful of candidates’ time (truly great managers rarely have much of it) and accurately separates the great from the good. Its interactive nature means sharing it publicly provides no advantage to future candidates. And in the spirit of open-source, I’m hoping others will find it useful in recruiting their own engineering managers.

Happy hiring.

Juan Pablo DellarroquelleJuan Pablo Dellarroquelle is the vice president of engineering at Medallia. Prior to joining Medallia, he worked as an engineer at Redmond Software, consulted to numerous Silicon Valley companies, and built Argentina’s first voice portal, which won him Intel’s Application of the Year Award.