Step 1: Before You Do Anything
But before you do anything, realize: “The questions don’t really matter. What matters first is a clear understanding of what you need,” says Soham Mehta, CEO of Interview Kickstart. Optimize your time by spending 80% to figure out what you need and 20% to craft the challenges.
The Questions Don’t Matter Yet
So, how do you know what you need? The two most common values of great engineers are:
- Technical knowledge
Algorithm challenges are best used to test the former and knowledge or tool-based challenges are great for the latter.
But what most managers don’t realize is it’s better to keep these values mutually exclusive. Testing for both at the same time puts far too many constraints, limiting your pool of talent. So, how do you know what you need? It largely depends on the size of your company:
Large companies, like Google and Facebook, are infamous for their algorithm and data structure challenges. When you have extensive teams, it’s better to hire for intelligence than knowledge. You likely have enough engineers to teach newcomers your tech stack. And smart people will be able to learn specific technologies pretty quickly anyway.
One 85-year-long organizational research study concludes: “cognitive ability (or intelligence) tests are the best predictor of success across fields.” Algorithm challenges, including ones that ask you to balance trees, are the programming equivalent of “cognitive ability tests” because they test for reasoning, problem solving and critical thinking skill.
What’s even more interesting is that the screening credentials commonly found on resumes—like education, age (or experience) and academic achievement — ranked among the worst predictors of success. Years of hosting coding challenges on behalf of high-growth companies reveal similar findings.
David Taylor, head of Sonos engineering, loves asking senior engineers data structure and algorithm questions because “if candidates shy away from that, you have a large red flag.” It’s a way to filter out folks who feel like they’re too good to roll up their sleeves and revisit the basics. Engineers should be continually interested in keeping themselves up to speed, in revising the fundamentals and taking on intriguing programming problems. Those are the people your best engineers want to work with.
Granted, the burden shouldn’t be entirely on the candidates. The onus is also on companies to properly prepare senior engineering candidates to ace these fundamental challenges. We’ll cover more on this in step 2.
Smaller companies—ones that can’t afford to wait for senior engineers to learn the tech stack they need—are most likely to focus on knowledge over intelligence. By crafting challenges that reveal technical knowledge in specific tools, you’ll attract engineers who can start hacking a new application on day one. So, it’s better to focus on knowledge-specific challenges if you’re a stealthy startup.
We'll go over how to craft powerful knowledge-specific challenges in Step 2.