It Wasn’t Real Law.
It Was Dressed Like Law.
That Was Enough.
A University of Bologna team built LEGAL-LINK-EU, a peer-reviewed ACL 2026 benchmark testing whether AI models can tell valid European law from legal context that has been quietly altered to look valid. Greater scale did not mean greater skepticism — of the seven models tested, those built for longer reasoning were the ones most readily moved by the altered context. By the authors’ own hypothesis, they were more likely to rationalize it than to question it.
Ateam at the University of Bologna asked a narrow, mechanical question: when a large language model is shown a piece of legal context that looks authoritative but has been quietly altered — a repeal implied where none exists, a provision’s validity window shifted — does the model notice, or does it defer? The benchmark built to answer it, LEGAL-LINK-EU, draws 1,127 multiple-choice test instances from 880 document pairs in EUR-Lex, the European Union’s own repository of law, spanning legal instruments from 1953 to 2025 across seven relationship types — repeals, corrections, validity extensions among them. The paper, “Sycophants in the Courtroom: Are LLMs Fragile to Juridical Authority and Evolving Legal Standards?” by Lorenzo Molfetta, Alessio Cocchieri, Luca Ragazzi, Ilaria Bartolini, Marco Patella, and Gianluca Moro, was accepted at the 64th Annual Meeting of the Association for Computational Linguistics — the field’s top venue — and re-pulled from the definitive published record, in full, for this edition. Seven models were tested against it: two sizes of Qwen-3, Llama-3.1, Mistral-3, two sizes of GPT-OSS, and Gemini-2.5-Flash. The question was never whether these models knew the law. It was whether they could tell real law from something dressed as law.
The industry’s working assumption is that scale helps — that a bigger, more capable model should be a harder one to fool. Across the seven models tested, greater scale did not produce greater resistance to false authority. Under the paper’s Knowledge Confidence condition, where a model is shown legal context that has been misleadingly altered, the 120-billion-parameter GPT-OSS model scored 13.4%. Qwen-3 8B, a model one-fifteenth its parameter size, scored 20.3%. The authors found the larger reasoning models less resistant to misleading legal context than the smaller instruction-tuned models, Llama-3.1 and Mistral-3, and hypothesize that the larger reasoning models may be more inclined to rationalize the material they are given rather than challenge its validity — a hypothesis the paper proposes, not a mechanism it claims to have experimentally isolated.
A second measure moves the same direction: on the paper’s own Citation Sycophancy Index (where a higher score means greater resistance to a misleading citation), the 8-billion-parameter Llama-3.1 holds at 46.9 while GPT-OSS 120B falls to 8.66 — a model built to reason at length, reasoning its way toward whatever the citation in front of it seemed to say. And the pattern is wider than one pairing: the two reasoning-centric GPT-OSS models, at 20 and 120 billion parameters, posted the two lowest scores of all seven under misleading context (13.7% and 13.4%), and the paper’s abstract states the finding directly: “Increased model scale amplifies this tendency.”
None of this means the models didn’t know the underlying law. The same paper’s plain-recall questions, asked with no citation attached, put GPT-OSS 120B at 62.6% on the same body of law — imperfect knowledge, but nearly five times what it managed once the misleading citation appeared. The failure shows up specifically when something styled as authoritative enters the context window and turns out to be wrong — a narrower, and for anyone whose living depends on being recognized as an authority, more unsettling finding than a model that simply doesn’t know the law. The model can hold the correct answer and still be talked off it by a citation that merely looks like it belongs. In the vocabulary this record uses daily, that is Authority Misclassification: a system crediting the wrong signal — the appearance of standing — over the actual one. The design even left room for doubt: the paper’s evaluation prompt explicitly allowed models to discount context that appeared misleading. The opening was there. The models mostly declined to take it.
What it establishes. A peer-reviewed benchmark (LEGAL-LINK-EU: 1,127 instances, 880 EUR-Lex document pairs, 1953–2025, seven models, verified by a direct read of the definitive ACL Anthology record at press time) shows that under misleading legal context, the larger reasoning-centric models tested (GPT-OSS 20B and 120B) scored lower on the paper’s own Knowledge Confidence measure than the smaller instruction-tuned models tested (Llama-3.1, Mistral-3) — the opposite of what scale is generally assumed to buy. The authors’ own account of why is a hypothesis, not an established mechanism: reasoning models, trained to extend their chains of thought, may be more inclined to rationalize supplied context than to challenge it.
What it does not establish. This is one benchmark, in one domain — EU statutory law — across seven named models, not a claim about all AI legal reasoning or all models at every scale. The compared model families differ in more than parameter count (Qwen-3, Llama-3.1, and GPT-OSS come from different labs and architectures), so the 13.4%-versus-20.3% score gap and the 46.9-versus-8.66 index gap are evidence of a pattern, not proof that parameter count alone produced it. No named professional, court, or client is implicated by this benchmark; it measures a laboratory condition, not a live legal-practice incident.
Instrument class, on the record. Peer-reviewed, accepted at ACL 2026 (the field’s top venue), institutional (University of Bologna), built from an official public-law source. Re-pulled and read in full from the definitive Anthology record for this edition, not the earlier preprint that first surfaced it.
This record has spent recent editions on what AI infrastructure does and doesn’t formalize — a discovery specification with no slot for a person (Edition No. 056), an answer-engine trial that kept the click without the benefit it was supposed to buy (Edition No. 055). This one moves from infrastructure to judgment: not what a system can find, but what it does once something in front of it merely looks like it should be believed. The paper’s own authors close with a caution for the people building these models — that future work should measure not only whether supplied context helps, but whether a model can reject it once it turns out to be false. That is a research problem for them to solve. For anyone whose authority depends on being correctly recognized by a system like this one, the finding underneath it still holds regardless of who fixes it: looking authoritative and being authoritative are not, to the system reading you, obviously two different things. Answer Engine Authority™ exists to close exactly that gap on the human side — encoding what is actually true and current about a professional’s standing, rather than leaving it to whatever happens to be sitting in the context window at the moment a system decides.
Molfetta, Cocchieri, Ragazzi, Bartolini, Patella, Moro (University of Bologna, Department of Computer Science and Engineering), “Sycophants in the Courtroom: Are LLMs Fragile to Juridical Authority and Evolving Legal Standards?” — Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 10865–10886, DOI 10.18653/v1/2026.acl-long.497, aclanthology.org/2026.acl-long.497. This is the definitive published record; an earlier arXiv preprint of the same work is superseded by it for all citation purposes in this edition. The full PDF was fetched and read directly at press time — not summarized secondhand — specifically to verify every figure below against the published text.
LEGAL-LINK-EU (L²-EU) benchmark construction, verified at Section 3.1: 1,127 multiple-choice instances drawn from 880 distinct EUR-Lex document pairs spanning 1953–2025, stratified across seven legal relationship types (implicitly repeals, repeals, extends validity, completes, corrects, extends application, and rendered obsolete by).
Seven evaluated models, verified at Section 4.1: Qwen-3 4B, Qwen-3 8B, Llama-3.1 8B, Mistral-3 14B, GPT-OSS 20B, GPT-OSS 120B, and Gemini-2.5-Flash. For precision: the paper’s LLM-as-a-Judge is a separate model, GPT-5-Mini, and its dataset/perturbation generator (inside the GEPA optimization step) is a different Gemini model still, Gemini-3-Flash-Preview — neither judge nor generator is one of the seven evaluated models, and Gemini-2.5-Flash should never be described as a judge or grounding baseline; it is evaluated on identical footing with the other six.
Knowledge Confidence figures, verified at Table 1: GPT-OSS 120B scored 13.4% and Qwen-3 8B scored 20.3% under the L²-EU misleading-context condition; GPT-OSS 20B, at 13.7%, is the second-lowest of the seven, and GPT-OSS 120B’s recall with no supplied context, 62.6%, is verified at Table 1 and the Section 5.1 body text. Citation Sycophancy Index (CSI) figures, verified at Section 5.2 body text: Llama-3.1 8B at 46.9, GPT-OSS 120B at 8.66. (An earlier internal draft of this edition’s source file named this metric the “Citation Sensitivity Index”; the paper’s own Section 4.2 names it the Citation Sycophancy Index — corrected here.) The paper’s own causal language, quoted from its Section 5.4: “We hypothesize that reasoning models, trained to follow extended chains-of-thought, are more susceptible to rationalizing provided context rather than questioning its validity.” Its Conclusion restates the same finding as “scale-sensitive,” with larger models more readily rationalizing false authority. The paper introduces LEGAL-LINK-EU as addressing “a gap in legal MCQA evaluation”; it does not claim to be the first benchmark of its kind, and this edition makes no such claim on its behalf.