Most pages ranking for job simulation assessment are written by vendors, and I'll say up front that this one is too. What I noticed reading them is that the phrase has stopped meaning one thing. I've seen it used for a multiple-choice quiz with a paragraph of context on top, a solo spreadsheet exercise, a role play with a hiring manager, an exercise where the facts are spread across several people, and one where you then have to defend the call to someone who disagrees. Those measure very different things. If you're a recruiter with open roles in sales, customer success, product and engineering at the same time, you'll get pitched all five under one name. This piece gives you a ladder to sort them and five questions to ask before you buy.
What the label covers, and what it hides
A job simulation assessment is a pre-hire exercise that puts a candidate inside some version of the job and scores what they do. The US Office of Personnel Management describes work samples and simulations as tasks that mirror what employees actually do, and that definition is fine as far as it goes. The problem is the phrase "some version". A version can keep almost everything the job has, or it can keep a paragraph of scenery and throw the rest out.
Here's my bias, so you can discount accordingly. Skillvee is a 60-minute "day at work" simulation that replaces the recruiter phone screen and the first interview round. The candidate solves a real challenge, talks to AI teammates to get the information they need, and defends their decisions on a call, while the screen is recorded. We sit at the top of the ladder below, which is exactly why I think the ladder matters, and why you should check whether I've drawn it fairly.
The fidelity ladder: five rungs of job simulation
Each rung adds one thing the real job has that the rung below it removes. Nothing on the ladder is useless. They just answer different questions, and the mistake I see most often is buying a lower rung while expecting the answer from a higher one.
| Rung | What the candidate does | What it adds | What it measures | What AI does to it |
|---|---|---|---|---|
| 1. Situational judgment | Reads a scenario, picks or ranks responses | A work-shaped context | What they say they'd do | A model picks the textbook answer instantly |
| 2. Solo work sample | Gets a complete brief, produces an artifact alone | Actual output | Whether they can produce the deliverable | A competent draft appears in a minute if the brief is complete |
| 3. Scripted role play | Talks to one counterpart playing a fixed part | Another person | How they handle a live conversation | Harder to outsource live, though real-time copilots are narrowing that |
| 4. Multi-party simulation | Gathers information spread across several people and sources, then does the work | The information isn't in the brief | Whether they go and get what's missing, and use it | AI becomes a tool inside the work rather than a way around it |
| 5. Defended simulation | Rung 4, then explains and defends the call to someone who pushes back | Accountability for the decision | Whether they understand what they built and what it gives up | You hear whether the thinking was theirs |
The jump that matters most is from rung 2 to rung 4. Almost nobody in a white-collar job gets a complete brief. The deciding fact is sitting with a colleague, in an old ticket, or in a number nobody has checked, and most of the job is noticing the gap and closing it. I wrote about this at length in work sample test examples, under the name of the missing-information test. A rung-2 exercise has removed that part of the job on purpose, which made sense when producing the artifact was the hard bit. It isn't any more.
Situational judgment test vs job simulation
A situational judgment test is a legitimate format with a long history. It's cheap to run at volume and it's consistent. It tells you whether a candidate recognizes the sensible response when four of them are laid out on a page. What it can't tell you is whether they'd have thought of it unprompted, halfway through a messy afternoon, with a customer waiting. Those are different skills, and the second one is usually what the hiring manager means when they say they want to see someone work.
I don't think vendors selling rung 1 as a job simulation are being dishonest. The word has drifted, and a scenario-based quiz is a simulation in a loose sense. But when a hiring manager hears "job simulation" they picture rung 4, so if you buy rung 1 you'll spend the next quarter explaining why the scores said nothing about how a candidate handles that messy afternoon. Ask to take the assessment yourself before signing. You'll know which rung it is in about five minutes.
Five questions to ask before you buy a job simulation test
These work on any vendor, including us. Each one places the product on the ladder, and none of them needs a technical background to ask.
- Is the answer in the brief? If a candidate could finish the exercise with nothing but what they're handed, it's rung 1 or 2, however realistic the scenery looks.
- Who can the candidate talk to, and does that counterpart behave the same way for every candidate? A live hiring manager improvising the buyer is rung 3 run without a script, and it's also the least consistent part of most processes, because the fourth candidate of the day gets a tired buyer who read the CV first.
- Can the candidate use the tools they'd use on the job, AI included, and what gets scored when they do? A test that bans AI is measuring a job that doesn't exist. A test that allows it without scoring how it's used can't tell AI leverage from copy-paste.
- What's the evidence behind a score? You want to click on a rating and see the moment that produced it: the question they asked, or the decision they made after the answer came back. A number with no trail is hard to defend to a hiring manager and harder to defend to a rejected candidate.
- Does the candidate ever have to defend a decision to someone who disagrees? This is rung 5, and it's the one that catches work the candidate didn't really do.
If a vendor answers all five well, you're looking at rung 4 or 5. If they answer the first one with "the scenario is very realistic", you're probably looking at rung 1.
Which rung for which role
You don't need the top rung for everything, and I'd rather say that than have you find out the expensive way.
For high-volume roles where the work is procedural, a rung 1 or 2 screen is often the right call. It's cheap and consistent, and the job doesn't hinge on gathering missing information. For a sales role, the conversation is the job, so I wouldn't go below rung 3, and the buyer has to behave identically across candidates or the comparison is meaningless. The wider tool choice is in sales hiring assessment tools. I've written the fixed-buyer version out in sales role play interview scenarios. Customer success and product sit at rung 4 at minimum, because the whole job is pulling the real problem out of people who don't volunteer it; there's a worked example in the customer success interview case study. Engineering and data roles belong at rung 4 or 5 now that a solo take-home can be finished by a model, which is the argument I make in the take-home interview alternative piece.
Across functions, the more of the job happens between people, the higher up the ladder the assessment has to sit to see it.
Where this gets hard
Climbing the ladder costs something, and you should know what before you commit.
Cost goes up with every rung. OPM's own guidance flags that simulations are expensive to build and to run with trained observers, and that's true. A homemade rung-4 exercise burns an hour of a colleague's time per candidate. That's fine for the six people you'd have phone screened anyway, and it doesn't scale to two hundred applicants. It's a big part of why our counterparts are AI teammates, and it's also a reason you might reasonably stay on a lower rung for some roles.
Consistency slips as well. A person running a rung-5 defense pushes back differently on the sixth candidate than the first, and a script only fixes part of that.
And some candidates find the higher rungs strange at first, particularly talking to AI counterparts. What helps is saying plainly what's going on: the brief is incomplete on purpose, the people who know more are reachable, and asking them is part of the job. In my experience the candidates who are good at the work relax once they hear that, though I'd call that an observation rather than a finding.
If you want to see what rung 5 looks like on one of your own open roles, that's the thing worth an hour of your time.