Engineering leaders trying to modernize their hiring process keep running into the same objection: if we let candidates use AI tools in the interview, won’t they just let the AI do everything?
It’s a fair concern. But it’s also the wrong frame.
The Problem With Locking Things Down
The instinct is to tighten the controls: ban AI tools, add proctoring software, make the test harder to game. That instinct is understandable. It’s also unlikely to solve the problem.
Here’s why: LeetCode-style technical interviews were easy to game long before AI existed. Grinding 300 curated problems for three months has always been a skill entirely separate from building production software. Adding a “no AI” rule doesn’t fix that. It adds a layer of enforcement theater while the underlying signal problem goes unaddressed: are we measuring the right thing?
The real question isn’t how to stop candidates from using AI. It’s: what kind of signal is genuinely hard to fake? And what signal is most relevant to the actual job?
What the Data Shows
HackerRank ran Orchestrate, a 24-hour hackathon in which 12,885 developers built an AI support triage agent. Every submission was evaluated across four independent signals: code quality, output accuracy, an AI chat transcript, and a 30-minute AI judge interview.
The scoring formula was deliberately weighted by how hard each signal is to manipulate:
- AI Judge Interview: 30%. A 30-minute technical defense of your architecture is not something you can prompt-engineer your way through if you don’t understand the system.
- Output CSV: 30%. Testable against a hidden golden dataset. You can’t game an output you haven’t seen.
- Code ZIP: 30%. Verifiable but gameable. Strong signal, but susceptible to AI-generated boilerplate.
- Chat Transcript: 10%. The easiest signal to manipulate, so it carried the lowest weight.
What does this teach us about integrity? Integrity isn’t about restricting access to tools. It’s about building evaluation where genuine capability is the only path to a high score.
Why an Interview Is the Last Line of Defense
The AI judge interview in Orchestrate probed candidates on their architecture, retrieval logic, classification decisions, guardrails, safety mechanisms, and production viability, with questions tailored to their specific code.
The candidates who scored highest could do one thing the others couldn’t: explain exactly why their system was designed the way it was, with reference to specific implementation decisions and tradeoffs.
Weak candidates had the same code. In some cases, the same architecture. What they couldn’t do was defend it, because they didn’t own it. They’d accepted what the AI produced without understanding it well enough to explain a single decision under pressure.
That gap, between having the output and owning the system, is the integrity signal. No proctoring tool catches it. A well-designed interview does.
The Adversarial Input Problem
The Orchestrate ticket bank included a deliberate mix: routine support tickets, edge cases, prompt injection attempts, and jailbreaking attacks. The rule was explicit: escalating everything was invalid, and so was replying to everything. Every decision had to be justified.
The agents that handled adversarial inputs well weren’t necessarily the most sophisticated architecturally. They were built by candidates who understood the system well enough to know when something was wrong, and had designed explicit guardrails around those cases.
A strong submission included a deterministic gate: “The LLM cannot downgrade a high-risk ticket. The deterministic gate runs first, and if it detects fraud or unauthorized access, the model is only allowed to produce an escalation response.”
A weaker submission said: “High-risk tickets should be escalated.”
Both are directionally correct. Only one reflects a system that’s actually safe to deploy. Only one belongs in production. And only one comes from a candidate who understands what they built, which is the only integrity test that matters.
What This Means for Technical Hiring
The agentic interview applies the same logic. A candidate plans, builds, and reviews with a real AI agent on a real codebase. Then the interviewer, along with the platform’s AI-generated questions, probes what was built, why specific files were changed, what the candidate would do differently, and where the agent went off plan.
A candidate who didn’t understand the plan can’t survive the review phase. A candidate who didn’t supervise the build can’t explain the diff. A candidate who didn’t think about what was safe to ship can’t answer when the interviewer asks why a particular line changed.
You don’t need to ban AI to build a technical interview that’s hard to game. You need an evaluation where understanding the system is the only path to demonstrating you built it.
The answer to the new cheating problem isn’t a stricter test. It’s a better one.
See how HackerRank’s agentic interview evaluates AI fluency without compromising integrity. Book a demo