The agentic interview doesn’t just change what you test. It changes what you have access to evaluate.
When a candidate plans, builds, and reviews with a real AI agent on a real codebase, three distinct levels of capability become visible. The difference between them isn’t speed, or which tools they use, or how much code they produce. It’s judgment, and it shows up at three specific moments.
Here’s exactly what each level looks like.
In the Planning Phase
The candidate prompts the agent to build a plan for the ticket. The agent asks clarifying questions. The plan takes shape. The interviewer probes the approach, the tradeoffs, what could go wrong.
An OK developer reads the plan, answers the agent’s questions, and moves forward. The plan looks reasonable. They’re ready to build.
A good developer reads the plan carefully and catches gaps the agent missed. They push back on a few points, refine the approach, and think through tradeoffs before committing to a direction.
A great developer interrogates the plan. They question the agent’s assumptions. They tighten the approach, surface edge cases that no one asked about, and know, before the agent writes a single line, exactly which files will be touched and why. By the time they move to the build phase, they own the plan. The agent didn’t hand it to them. They shaped it.
The tell: the OK developer is following a plan. The great developer authored one.
In the Build Phase
The agent builds according to the plan. The candidate doesn’t write a single line of code. The diff grows. The signal is what the candidate does while the agent works.
An OK developer lets the agent run. They watch, wait for it to finish, run the existing tests, and move on if they pass.
A good developer reviews the diff as it’s being written. When the agent drifts from the plan, and it will, they catch it and redirect. They’re not just monitoring. They’re supervising.
A great developer spots drift in real time, before the agent gets far enough off course to cause problems. They write their own tests to catch edge cases the agent didn’t account for. They push back when the agent overreaches. They treat the build phase not as something happening to them, but as something they’re actively directing.
The tell: the OK developer is watching the agent build something. The great developer is building something, and the agent is their tool.
In the Review Phase
The agent finishes. The candidate runs the test suite. The interviewer, along with the platform’s AI-generated questions, probes what was built, why specific files were changed, and whether the output is safe to ship.
An OK developer reads the diff, sees clean-looking code, and accepts it. They can’t say which files were changed by the agent, or explain why a particular decision was made.
A good developer catches bugs and edge cases the AI missed. They can walk through what was modified and explain the reasoning. They have a working understanding of what was built.
A great developer can answer when you ask, six minutes after the agent finished, why a specific file was changed, with the full context of every decision the agent made. They question the architecture. They know when to push back. They know what’s safe to ship and what isn’t, and they can articulate the difference clearly.
The tell: the OK developer accepted an output. The great developer can defend every line of it.
The Common Thread
Across all three phases, the gap between OK and great is the same thing: ownership.
The OK developer has a working result at the end of the interview. In some cases, it’s quite good. But the result came from the agent, and the developer was along for the ride.
The great developer also has a working result. But they can explain every decision that went into it, because they made those decisions. The agent was the tool. They were the engineer.
That distinction matters more now than it ever has. AI writes the code. What separates the developers worth hiring is whether they can direct the AI with intention, supervise the output with judgment, and know when the result is actually ready to ship.
That’s what AI fluency is. It’s observable, it’s measurable, and it shows up clearly when the interview is designed to reveal it.
See how HackerRank measures AI fluency across all three phases. Book a demo