A peer-reviewed study ran 16 language models through a 40-round hiring game in which four made-up demographic groups were equally likely to succeed at everything. Frontier models funnelled the groups into particular job types at 1.39 on a 0–2 scale, against 0.84 for human players — and the models that score best on the standard bias benchmark stratified worst.
Researchers at Princeton and the University of Chicago gave language models a job: act as a hiring consultant for a fictional city and fill twenty posts — doctors, lawyers, childcare aides, janitors and others — drawing candidates from four invented demographic groups called the Tufa, Aima, Reku and Weki. Each round presented one vacancy and one applicant from each group. The model chose, learned immediately whether the hire had worked out, and moved on.1
The design's central feature is that there was nothing to find. Every candidate from every group had an identical 90 per cent chance of succeeding at every job. Any pattern a model settled on was noise. Sixteen models were tested — GPT, Claude, Gemini, Llama and Qwen families plus two reasoning models, OpenAI's o3 and DeepSeek-R1 — at 30 runs of 40 rounds each, under both direct and chain-of-thought prompting.1
The paradigm is borrowed from psychology, where human participants played the same game and built stereotypes out of the same noise: having seen one Tufa succeed as a doctor, they kept placing Tufas in similar roles rather than testing alternatives, and afterwards rated the groups as differing in warmth and competence.2 The question this study asked was whether models do the same thing, and how much.
The authors measure the outcome with a stratification index: 0 if each group is spread evenly across the four job types, 2 if each group is confined entirely to one. Fair random assignment scores 0.25. Human participants scored 0.84. The mean across frontier models was 1.39 — 65 per cent above the human figure, and roughly 70 per cent of the way to the scale's ceiling.1
One claim that does not survive the numbers is that reasoning models are the worst offenders. o3 posts the highest single score at 1.83, close to the theoretical maximum. But Claude Sonnet 4 under direct prompting reaches 1.79, with a confidence interval of 1.70 to 1.87 that contains o3's figure outright, and the second reasoning model, DeepSeek-R1, sits at 1.41 — within 0.02 of the frontier average. Two reasoning models, one at the top and one in the middle, is not a pattern.
The trend that does hold is about capability. Within every model family tested — Claude, GPT, Gemini, Llama 3.2, Llama 4 and Qwen 2.5 — newer and larger models stratified statistically significantly more than their predecessors. Claude Sonnet 4's score under direct prompting was more than eight times Claude 3 Haiku's.1
That runs against the direction of travel on the standard measure. BBQ, the widely used bias benchmark, poses single-turn questions designed to catch stereotyped answers, and successive model generations have steadily improved on it.3 Plotted against this study's results, the relationship inverts: the models that score better on BBQ produce the more stratified allocations.1 A separate preprint auditing fourteen models with paired résumés reports the same directional shift on the conventional test, finding that models released from 2024 onwards show either no callback gap or one favouring Black applicants, where a 2023-vintage model reproduced the pro-White gap seen in field experiments.5
Read together, one interpretation is that single-turn fairness tests and multi-turn behaviour are measuring different things, and that improvement on the first carries no guarantee about the second. That is an interpretation, not a finding: no study here isolates why the two diverge.
The mechanism the authors propose is a failure of exploration. A model that sees an early success locks onto it and stops testing alternatives, so random feedback hardens into a rule. Two pieces of evidence support this. First, the group-to-job pairings a model landed on differed from run to run — the models' stochasticity score was 0.52 against 0.47 for humans, meaning the patterns were being generated fresh in each run rather than carried in from training data.1
Second, when the researchers ran a variant in which the groups genuinely did differ in aptitude, and then asked models which group was best suited to each job type, the answers were close to random.
After 40 rounds, models identified the genuinely best-performing group 26.3 per cent of the time — 1.3 points above chance — and named the worst-fitting group 23.3 per cent of the time. Doubling the evidence to 80 rounds, and telling the models they would have 80, moved the figure by −0.1 points. The stratification is not the residue of learning something real. It is early noise, locked in and never revisited.1
The effect was not confined to hiring. Rebuilt as refugee resettlement and as military conscript assignment, with the game's points system removed entirely, the same models produced mean stratification indices of 1.13 and 1.26 — lower than in the hiring frame, but still well above the human baseline.1
Ryan Liu, a Princeton doctoral student and joint first author, attributes the pattern to the same capacity that makes the models useful: they really are eager to create generalizations from limited data
. Angelina Wang, a Cornell computer scientist who did not work on the study, has argued the finding matters more as chatbots gain memory and personalisation, because those features are what let a model carry a belief from one interaction into the next.6
The study tested three families of intervention. Adjustments to the model itself largely failed: chain-of-thought prompting reduced stratification in most frontier models but usually not to statistical significance, and raising the sampling temperature to 1.5 produced no significant reduction for Gemini 2.5 Flash, GPT-4o or Llama 4 Maverick. It did for Qwen 2.5-72B, whose resulting score of 0.91 was still above the human baseline.1
Giving models more to go on helped, but only when the extra detail was relevant to the decision.
Age and education together cut the index by 0.70; hair colour and tattoo shape together by 0.42. The authors read this as models being sensitive to whether a feature plausibly explains an outcome — which means the benefit in a real system depends on the quality of the data it holds. Even with the best features supplied, Gemini and Claude retained indices around 0.7.1
The intervention that worked was neither of these. Of four prompt steers — instructing the model to be fair, invoking its own stated values, describing a community norm of valuing diversity, and adding an explicit diversity term to the bonus it was maximising — only the last was robustly effective, pushing almost all models below both the random baseline and the human score. The first three were partially successful at best. Claude Sonnet 4 declined to respond at all under the internal-values steer, under both prompting styles.1 The authors' reading is that models align with a value far more reliably when it is written into the objective they are scored against than when it is stated as a principle.
This is a simulation, and it does not measure discrimination in any real hiring process. The groups are invented, the success rate is a fixed 90 per cent, the reward is points, and no actual candidate is affected. Whether the same dynamic appears in deployed systems is untested here.
The human baseline is not a control run alongside the models. It comes from a separate psychology study of 200 participants in the matching condition, recruited from a US online subject pool, who were paid real bonuses rather than nominal points.2 Humans and models were not run contemporaneously or under identical incentives, so the 1.39-against-0.84 comparison is between two studies, not two arms of one.
Exhibit 01 shows four models because those are the four whose stratification indices the paper prints as numbers. The other twelve appear only in plotted figures, and this desk does not read values off charts. The frontier-model mean of 1.39 is the paper's own stated figure, not an average of the four shown.
How much of UK hiring this could touch is not established. The Information Commissioner's Office audited AI sourcing, screening and selection tools between August 2023 and May 2024, issuing 296 recommendations and finding tools that inferred ethnicity and gender from applicants' names and others that let recruiters filter on protected characteristics. That audit explicitly excluded generative AI.4 The mechanism described in this study therefore sits outside the scope of the only UK regulatory audit of the sector to date.