At a recent HackerRank customer advisory board in London, an engineering leader said something that stopped the room.
“We need to find people who know what good looks like. But the people we have now don’t know how to evaluate it. It’s a contradiction in terms.”
He wasn’t wrong. And nobody in the room disagreed.
Across the table were hiring leaders from organizations with mature, global recruiting functions. All of them said some version of the same thing: the frameworks they’ve relied on for a decade no longer feel like reliable signals. And they’re not sure what to replace them with.
The definition of a developer just changed
For roughly 20 years, the software development lifecycle gave us a tidy map: gather requirements, design systems, write code, test, deploy, maintain. Hiring followed the map. Can this person solve algorithmic problems? Can they write functionally correct code from a detailed spec? Those were the questions worth asking.
They aren’t anymore.
AI can produce functionally correct code from a spec faster and more reliably than most engineers. That doesn’t make engineers obsolete. It changes what they need to do. The role has shifted from builder to orchestrator. An individual contributor today is effectively an engineering manager of agents: directing, reviewing, correcting, and extending AI-generated output. The build phase, which interviews have been designed around for 20 years, is increasingly the part AI handles.
What matters now is what comes before and after. Can this person plan clearly enough that an agent can execute? Can they review what comes back, catch what’s wrong, and push the output further? Those are different skills from what LeetCode measured. And they’re harder to assess.
“AI fluency” is the thing everyone wants and nobody can define
Ask any engineering leader what they’re hiring for now and you’ll hear the same two words: AI fluency. Ask them what that means, and the conversation gets complicated fast.
In London, one program manager at a global tech firm described being asked to build evaluation rubrics around AI fluency and finding that his own engineers couldn’t agree on what it looked like. “They’re extremely neurotic about this,” he said. “They want me to over-specify the requirements in ways I honestly can’t. They want so much more detail. And I don’t know.”
This isn’t a process problem. It’s a definition problem. Most teams are trying to assess a capability they can’t yet clearly describe. The instinct is to add more rubric, more criteria, more guidance. But when the underlying skill is still being defined in the industry, no rubric is going to resolve the ambiguity.
What does seem to be emerging: AI fluency is less about tool familiarity and more about judgment. Does the candidate check AI output or just accept it? Do they notice when the model is wrong? Do they know when not to use AI? Can they articulate the reasoning behind what the agent produced?
Those signals don’t show up on a standard take-home assessment. They surface in dynamic, interactive evaluation — the kind that requires back-and-forth, not just code submission.
The process is running on yesterday’s logic
Here’s what most technical hiring processes still look like: a recruiter call to check location, work authorization, and compensation expectations, followed by a static take-home assessment, followed by a set of interviews built around whiteboarding or pair programming with an IDE.
That structure was designed for a world where writing code was the core job. It does a reasonable job of filtering for people who can write code. It does almost nothing to assess whether someone can direct, evaluate, or improve AI-generated code.
The mismatch is showing up everywhere. Teams are running interviews designed for 2016 to fill roles that exist in 2026. They’re asking candidates to solve problems in 45 minutes without assistance — then expecting those same candidates to spend their working days collaborating with AI tools to produce better output than either could alone.
“Most of the interviewing we’re doing is interviewing for what the job looked like yesterday,” one senior engineering manager said in the room.
What’s actually working
A few patterns are emerging from organizations experimenting with newer approaches.
Requiring AI use, not banning it. Some teams have moved to interview formats where AI use isn’t just permitted — it’s mandatory. The evaluation isn’t whether the candidate can solve the problem. It’s how they work with AI to get there: what they prompt, what they verify, what they catch when the output is wrong.
Making problems unsolvable without AI. One company described deliberately designing a problem too complex to complete in one hour without assistance. The goal wasn’t to see if the candidate could finish. It was to observe how they used the tool under pressure — and whether their instincts were good.
Evaluating the bookends. Under a framework being adopted by a growing number of HackerRank customers, the plan phase and the review phase of a task carry more evaluation weight than the build phase. An engineer who plans clearly and reviews rigorously is demonstrating judgment. An engineer who just executes isn’t — even if the code works.
Treating fundamentals as a prerequisite, not a ceiling. Several leaders noted that deep CS fundamentals and AI fluency aren’t in tension — they reinforce each other. The engineers who get the most out of AI tools tend to be the ones who understand what the model is actually doing. Fundamentals aren’t going away. They’re becoming a prerequisite for using AI well, rather than the main thing being assessed.
The honest answer
Nobody has this figured out. That was the most consistent message across every conversation in the room.
What’s clear is that the window for using old methods to make new hiring decisions is closing. The candidates being evaluated today will be doing a fundamentally different job than the one the interview is designed to screen for. The process needs to catch up — not incrementally, but structurally.
The organizations moving fastest aren’t the ones that have the best answers. They’re the ones that have stopped pretending the old answers still apply.
HackerRank works with engineering and talent teams across EMEA and globally to modernize technical hiring for the agentic era. Learn more about next-generation assessment and interview frameworks at hackerrank.com.