Should you allow AI in coding interviews?

For most software roles, yes, as long as you can watch the work. Banning it leaves you guessing about who complied, and the guess is unreliable. Allowing it removes the question of whether someone used a model and replaces it with how well they directed one, which is closer to the job. The condition matters though. If you allow the tool but only see a final artifact, you have made the interview easier to pass and learned less than before.

What should you measure when candidates use AI in an interview?

Six behaviors, all observable in a recorded session: whether they frame the problem before prompting, what context they feed the model, how much of the first output they discard, how they verify what it gives back, when they stop asking and do it themselves, and whether they can explain the finished work with the assistant closed. Scored together these separate leverage from dependence.

What is the difference between AI leverage and AI dependence?

Both produce working output quickly, which is why they look identical early on. Leverage shows up as direction: the candidate holds a model of the problem, uses the assistant to move faster inside it, and notices when the output drifts. Dependence shows up when something goes wrong. The dependent candidate has no independent read on whether the answer is right, so a wrong answer becomes the new starting point instead of a thing to reject.

How long does an interview need to be to see AI leverage?

Longer than a first output. In the opening stretch a candidate who is directing the model and one who is copying from it produce similar-looking work. The difference appears at the first wrong answer, the first constraint the model ignores, or the first follow-up question. If the exercise never produces one of those, the format cannot distinguish the two no matter what the rubric says.

Do you need a new rubric to allow AI in technical interviews?

Yes, and this is the part teams underestimate. Most existing question banks were written on the assumption that producing the code was the hard part, and a good chunk of them are now one prompt from being finished. Allowing AI without rewriting the questions and defining what good use looks like turns the interview into an unscored exercise, which is worse than the ban it replaced.

Allow AI in Coding Interviews: What to Score

Canva now expects candidates in several engineering tracks to use AI tools in a technical interview, and Meta has said it will let candidates use AI in some job interviews. If you're about to allow AI in coding interviews, the policy argument is the part your team will spend three meetings on and the part that matters least. Saying yes is easy. What it costs you is a question you used to answer by accident: whether this person can actually do the work. Here are the six behaviors I'd score instead, and the failure mode nobody warns you about.

Should engineers use AI in interviews? That's the easy half

The reason banning models stopped working isn't that candidates got sneaky. It's that the ban was never enforceable in the first place, so what you had was a rule plus a guess about who followed it. I've written at length about why detecting AI cheating is the wrong question, and the short version is that the guess flags your careful candidates about as often as your dishonest ones.

So teams flip the policy. Fine. The trouble is what happens next, which is usually nothing. The rubric stays the same, the questions stay the same, and now everyone finishes them. A test that used to sort your pipeline stops sorting it, and the interview quietly becomes a formality that everybody passes.

That's the cost of the policy change, and it lands whether or not anybody voted for it. The difficulty of typing code was doing quiet work on your behalf for years. It sorted your pipeline without anyone having to design the sort, which is why nobody wrote it down as a thing the interview was measuring. Allow the tool and that goes away, and now you have to go get the signal deliberately.

Six behaviors that show real AI leverage

This is the framework I'd hand an interviewer on Monday. All six are things you can see in a recorded working session, and none of them require an opinion about what the candidate was thinking.

Do they frame the problem before they prompt? Watch the first ninety seconds. Some candidates restate the thing in their own words, name the constraint that makes it awkward, or ask you a question. Others put the ticket straight in the box. The ones worth hiring almost always spend a beat deciding what they're actually asking for.
What do they load into the context? This one is stark once you start looking for it. Strong candidates paste the real schema, the actual stack trace, the constraint that makes this problem theirs and not a generic one. Weaker candidates paste a tidied-up description and get back a tidied-up answer to a problem nobody has.
What do they throw away? The first output lands, and now you count what survives. Someone with judgment reads it as a draft with two good ideas in it and deletes the rest without agonizing. Someone without it starts editing around the edges of whatever they were given.
How do they check it? Running it, writing a quick test, poking the edge case they suspect, going and reading the actual docs. Or "it looks right," carrying the entire weight. If I could keep only one of these six, I'd keep this one, because it's the one that gets more valuable as the models get better rather than less.
Do they know when to stop asking? There's a moment in most sessions where the model starts going in circles. Some candidates notice, close the tab, and do the next bit by hand or come ask you. Others keep rewording the same prompt and burn twenty minutes on it.
Can they own it with the assistant closed? At the end, have them walk you through the finished work, including a part they didn't type. Cheapest of the six to run, and the most brutal. Four minutes, very hard to fake.

You don't need all six to be strong. Two and four carry most of the weight in my experience, and six is the tiebreak when a debrief is split.

Leverage and dependence look the same for ten minutes

Here's the thing that surprised me, and it's the reason a lot of AI-allowed interview formats don't work even when the rubric is good.

For the first stretch of a session, a candidate who's directing the model and a candidate who's copying from it produce output that looks about the same. Both are fast. Both have something running. If your exercise is short enough that it ends there, you've watched two people look competent and learned nothing that distinguishes them.

The difference shows up at the first thing that goes wrong. A wrong answer, a constraint the model ignored, a follow-up question that breaks the shape of what it gave back. The candidate with leverage treats that as information and adjusts. The dependent one has no independent read on whether the output was right in the first place, so the wrong answer just becomes the new foundation, and everything after it inherits the mistake.

The shape I keep running into goes something like this. The model hands back auth middleware that reads fine and happily treats an expired token as valid. It's a quiet bug. Nothing throws, nothing goes red, and the happy path works on the first try. The candidate who's going to do well writes the expired-token case within a few minutes, usually without being prompted, because they wanted to know whether the thing actually held before putting weight on it. The one who's struggling builds three endpoints on top and finds out near the end, when somebody asks why logging out doesn't log anyone out. Same starting output, same twenty minutes spent, and only one of them wrote something you'd merge. If your exercise stops before that bug can surface, both of those candidates score the same.

So the thing that decides whether any of this works is the exercise itself, more than the scoring guide people spend their time arguing about. It has to reliably produce a moment where the model is wrong and somebody has to catch it. If your exercise never generates one, the rubric is decoration. This is the same structural problem I ran into when I argued that the AI-resistant coding assessment is a mirage: you can't fix an observation problem by tightening the thing you're observing with.

Scoring AI leverage in hiring without collapsing into vibes

"Score their AI leverage" turns into an interviewer's gut feeling unless somebody writes down what good looks like before the interviews start. A few things that have kept it honest:

Score the six separately rather than as one impression. Otherwise an interviewer who liked the candidate quietly backfills a high number across all of them, and you come out of the debrief knowing they were "good" without knowing at what.

Write the anchors before anyone interviews. One line per behavior describing what weak, adequate, and strong look like. Behavior four anchors easily: weak accepts the output unexamined, adequate runs it, strong tests the case they suspect will break. Behavior one is genuinely hard to anchor, and I think that's diagnostic. If you can't describe what good framing looks like in a sentence, your interviewers aren't going to score it the same way either.

The other habit worth enforcing is notes about what happened instead of what the interviewer concluded. "Kept rewording the prompt on the auth bug for ten minutes, then went and read the library source" survives a debrief in a way that "good AI use" doesn't, because somebody can disagree with it.

Quick disclosure so you can weigh the bias: I'm building Skillvee, a 60-minute "day at work" simulation that replaces the recruiter phone screen and the technical first round. Candidates solve a real problem with AI available, talk to AI peers to get the information they need, and defend their decisions to an AI manager while the screen records. So of course I think watching the work beats scoring the artifact. Take the framework on its merits, and note that all six behaviors above are scoreable by a human with a recording and no product at all.

Where this gets hard

This isn't free, and I'd rather you hear the costs from me than run into them in month two.

The question bank is the big one. If your loop leans on exercises where producing the code was the hard part, most of them are now one prompt from finished, and rewriting them competes with everything else your team owes the quarter. Teams that allow AI without doing this work end up with an easier interview and a worse read, which is the outcome the ban was at least accidentally protecting them from.

Then there's attention. Scoring six behaviors from a working session costs more interviewer time per candidate than a pass or fail on a test does. That's a real budget line, and it's why I'd only run this on the round where you're already spending an engineer's hour.

The last one is calibration drift, and it's the one I'd watch after month two. Behaviors three and five depend on interviewers agreeing about what "confident deletion" and "knowing when to stop" look like, and two interviewers can drift a long way apart on those without noticing. Recalibrate on a shared recording every so often, the same way you would with any behavioral dimension.

None of that is a reason to go back to banning the tool. The ban has a worse accuracy problem and it degrades with every model release. But allowing AI changes what your interview measures, and teams that read it as loosening the standard are the ones who wake up a quarter later wondering why everybody passed. If you want a version of this that doesn't add interviewer hours, that's what we built the product to do.

You decided to allow AI in coding interviews. Now score these six things