A Voight-Kampff Test for Identifying Engineering Managers

When I joined Reddit in late 2016, I was faced with a unique challenge: tripling the size of the engineering organization within a year. At the time there were about 35 engineers on the team and a small number of “tech leads,” but there was little in the way of official management structure in engineering.

In fact, the very first question in the very first meeting with my new team was “what do managers even do?” I’d hear many variations of that over the coming weeks and I’m proud to say that with time and patience the question slowly changed. By week 4 it was no longer “what do managers do?” and more like, “what does a VP of Engineering do?” Progress!

Joking aside, we needed to scale and that meant getting more serious about our leadership structure and responsibilities. Eventually, I hit a critical mass of understanding with the team and they were willing to take the leap into identifying managers. But this meant I had to do something about all the “tech leads.”

Before going further, you may be asking yourself “what is a tech lead?” And that is a fantastic question because if you asked it to 5 reasonable people you’d get 14 and a half reasonable answers. That is to say, “tech lead” can mean a lot of different things and the role is highly dependent on where you work.

I remember way way back at the dawn of my career when my management skills were only first beginning bloom. My mentor Kevin took notice and said, “perhaps we should think about making you tech lead.”

“What’s that?” I asked.

His response, “Well… there’s not a clear definition, but the main idea is that by putting the word ‘lead’ into your job title you’ll automatically become a better manager.”

Although he said this jokingly, over the years I’ve never come across a more universally accurate description. That’s why I try to avoid the tech lead title in organizations I build. Do they manage people, manage scrums, manage tech, manage a part of the architecture? The answer to all of these questions is “yes, no, maybe, or it depends.”

It’s hard to scale an organization without clear responsibilities and I needed to find managers fast. For the purposes of this blog post, let’s start with the definition that “tech leads” are either proto-managers or proto-architects. They are ready to take on a broader set of responsibilities but unsure whether they want to oversee people or technology. How do we make the call?

Tech lead in. Manager or architect out.

I took inspiration from a classic film. If you’ve ever watched the movie Blade Runner you probably remember the Voight-Kampff test. At the risk of over-simplifying a cult-classic, Blade Runner is about a futuristic bounty hunter who is tracking down replicants (androids) hiding among the general human population. The idea behind the Voight-Kampff test is that you take someone who looks supposedly human, ask them a few weird questions, measure the responses, and determine whether they are truly human or a technological marvel.

Coincidentally, this is exactly what we needed to accomplish with our tech leads. The best Engineering Managers are interested in people first. The best Architects most fascinated by technology. Human or Android?

But unlike the movie, my approach was not to setup a lie-detector in the office and study tech leads in a lab environment. Instead, I integrated a few exploratory questions into normal 1:1 conversations over the course of several weeks.

Do you care more about people or technology?

Let’s start with the easy one. Engineering managers must be primarily concerned with people, processes, and execution whereas Architects are the stewards of the tech roadmap. That said, this question is a bit fuzzy because both groups care about people, process, and technology to a certain degree.

For that reason, this question is best at identifying outliers. You will occasionally find employees who sincerely think that good code alone solves all problems. These folks won’t make great people managers (and in the extreme, might propose your entire Python codebase be ported to Erlang over the weekend).

Conversely, you will occasionally find employees who believe that process and teamwork solve all problems. These folks won’t be the best stewards for your tech roadmap (and in the extreme, might nail pages from the “Agile Manifesto” to the front door of your office).

An executive asks for a feature to land by a set deadline, what are your thoughts?

Managers will try to figure out how to balance scope, resources, processes, and time to determine when a product can be delivered. Depending on the needs of the organization, they may also understand that some of those variables are unmovable. For example, is the exec asking for a deadline because of a holiday ship date?

In my experience, Architects tend to fall more into the “it’s done when it’s done” camp — not because they have a disregard for deadlines but because they can foresee the longer-term pain (tech-debt and refactoring) that can be introduced by date-driven mentality. Another pattern I’ve seen is the suggestion to solve urgent problems by forming v-teams, which is a great Architect superpower when deployed judiciously.

A PM goes to two of your engineers and asks them to immediately start working on a new feature, what do you do?

It’s great when the thing you are supposed to be responsible for is right there in the job title. Painters paint. Writers write. Engineering managers manage engineers. And management means setting the expectations for what people work on, how they do it, and whether it’s of acceptable quality.

In larger organizations, an EM will partner with their PM to understand product requirements but will not delegate engineering responsibilities to them (that said, in a smaller organization you may have program/project managers who do take on some of those tasks.)

When faced with this question an EM would think about what’s on the backlog, what’s in-flight, schedule impact, etc., before agreeing to the work. The great Architects I’ve worked with love being pulled in to consult on a new system. Because they are often brought in to think about problems long before people and resources have been fully committed, they wouldn’t see this work as a “interrupt” but more as an opportunity to make sure the right tech path will eventually be chosen.

You spend one day a week promoting your open position LinkedIn but no one replies. How do you feel?

This may be a surprising question but it was one of the most valuable. Since I was rapidly scaling the organization I knew that I’d be asking my managers to spend a lot of time promoting their jobs on LinkedIn and interviewing candidates.

Among other things, managers are responsible for attracting, retaining, and developing talent. A good EM will understand that building the team is part of the job and that the team members have to come from somewhere.

I suppose the best response I’ve gotten to this question from an Architect whom I highly respect was, “I could spend a day a week hiring, but wouldn’t you rather have me working directly with people who are already here to make them stronger engineers?”

Final Thoughts

As for the Voight-Kampff test for engineering managers, I like to the think the approach turned out well. About 90% of the managers I identified are still working in that role. All of the architects are still on board. Expectations and responsibilities immediately became much clearer than under the old “tech lead” system. Eventually many of the test questions were folded into a more scalable hiring process by working with the HR team.

Additionally, formally introducing an Architect role allowed us to create a career ladder that lets individuals rise to the highest levels of the organization via technical contributions. This is important because it should be entirely possible for people to grow in scope whether they have chosen to become people or technology managers.

Finally, observant readers may ask, “Hey at my company the tech lead role sounds an awful lot like your expectations for Architects. Did you really need to eliminate the tech lead title?” Perhaps not, but remember that I needed to move the organization quickly. You move fastest when the direction is extremely clear. Removing the old tech lead title granted an opportunity to define exactly what was expected from future leaders.

Hope you enjoyed the read! If so, please share it with a friend

rock on
-nick