Engineers today don’t write code line by line. They plan, prompt, and manage AI agents to do the work. The role has fundamentally changed.
But walk into most technical interviews happening right now and you’ll find candidates typing code manually into a plain editor, reversing linked lists from memory, stitching frameworks together by hand. Mechanics that an AI agent can perform in seconds.
The interview hasn’t caught up with the job. And that gap has a cost: good engineers, the ones who know how to orchestrate agents effectively, are being filtered out by a test that measures something they no longer need to do.
Fixing this requires changing three things.
1. The Task Has to Change
From: LeetCode-style puzzles and algorithm challenges.
To: Real customer tickets on real codebases, where the candidate plans, builds, and reviews with an AI agent.
The difference isn’t cosmetic. A LeetCode problem has one correct answer, a timer, and no context. A real engineering ticket requires understanding a codebase, diagnosing a bug, making decisions about what to change and what not to touch, and knowing when the solution is actually ready to ship.
In the agentic interview, a candidate receives a ticket the same way they would on day one of the job. The codebase is real. The bug is real. The agent is available from the start. The interviewer watches everything live.
That’s the task developers actually do. The interview should test it.
2. The Evaluation Has to Change
From: Functional correctness. Did the tests pass?
To: AI fluency. How effectively did the candidate work with the AI agent across every phase?
Correctness used to be a reliable signal because writing correct code was hard. Now AI writes the code. What’s scarce is the judgment to know whether the code is worth shipping.
AI fluency measures the capability that actually matters:
- In the planning phase: Did the candidate set clear constraints, pressure-test the approach, and catch gaps before the agent started building?
- In the build phase: Did they supervise the agent, notice when it drifted from the plan, and redirect it, or did they let it run and accept whatever it produced?
- In the review phase: Can they explain what was modified and why? Do they know what’s safe to ship and what isn’t?
AI fluency is measurable. It’s generated automatically from the interview transcript, scored across all three phases. Correctness is still measured; it’s just no longer the headline signal.
3. The Experience Has to Change
From: A simple code editor, often one the candidate has never used before.
To: An agentic development environment (ADE) that mirrors the tools candidates actually use at work.
An ADE includes support for the latest AI models (Claude, GPT, Gemini), a terminal, a real codebase, and an interface that feels like Cursor, Claude Code, or Codex. No context switch. No toy problem. No asking a senior developer to mentally translate what they’re watching into “how this person would actually perform on the job.”
The format is harder to standardize than a LeetCode test, and judgment is messier to score than pass/fail. But the tradeoff is worth it. Companies hiring this way get developers who can orchestrate agents, not developers who can ace a test that has nothing to do with their actual job.
The Common Thread
All three changes point to the same shift: the scarce resource in software development is no longer the ability to write code. AI handles that. The scarce resource is the judgment to know what’s safe to build, how to direct the tools that build it, and when the output is ready to ship.
If you’re still measuring the first thing while trying to hire for the second, you’re not just using an outdated interview. You’re making systematically wrong hiring decisions.
The task changed. The evaluation needs to change with it. And the environment candidates interview in needs to reflect the environment they’ll actually work in.
That’s what hiring for the agentic era looks like.
See all three changes in a live agentic interview. Book a demo