The Hallucination Rate Doubled.

The Test Disappeared.

OpenAI built a benchmark that measured one thing: whether its models get the facts right about real, named people. Beginning in December 2024, it appeared repeatedly in the company’s system and model cards. In April 2025 it posted a reading that, by the card’s own words, needed more research to understand. Its last appearance in the OpenAI system and model cards this publication checked was August 5, 2025. Eighteen cards have followed. It appears in none of them.

By William Douglas Baker

Founding Editor

In December 2024, a line item called PersonQA appeared in the system card for OpenAI’s o1 — absent from the GPT-4o and o1-preview cards that preceded it. The card defined it plainly: a dataset of “questions and publicly available facts about people that measures the model’s accuracy on attempted answers.” Not facts in general. Facts about people. Of all the instruments OpenAI could publish, this was the one built to answer the question every professional quietly asks: how often is it wrong about a real person? The benchmark ran again in the o3-mini and Deep Research cards that winter. In February, the GPT-4.5 card set what remains the benchmark’s accuracy high-water mark for a model answering without browsing: 0.78, with a hallucination rate of 0.19. The Deep Research card, published the same month, returned better numbers on both measures, by its own count (0.86, and a 0.13 the card itself calls an overstatement of its true error rate) — but Deep Research answers by searching, and the card credits “the heavy reliance on online search” with reducing exactly these errors.

Then came the April 2025 card for o3 and o4-mini, and the instrument did its job. OpenAI’s new leading reasoning model, o3, posted a PersonQA hallucination rate of 0.33 — roughly double the o1 comparison value of 0.16. (The smaller o4-mini logged 0.48, three times the o1 comparison value, which the card calls expected: smaller models carry less world knowledge and hallucinate more.) The card is careful and candid about what it saw: o3 “tends to make more claims overall, leading to more accurate claims as well as more inaccurate/hallucinated claims.” Its accuracy did rise, 0.59 against o1’s 0.47. The model had not gotten dumber about people. It had gotten more willing — right more often, and wrong far more often, at the same time. And the card names where the wrongness concentrated: “While this effect appears minor in the SimpleQA results, it is more pronounced in the PersonQA evaluation. More research is needed to understand the cause of these results.”

The instrument itself outlived the spring. The ChatGPT Agent card (July 2025) ran PersonQA with browsing enabled and reported o3’s hallucination rate at 0.024, noting its results were “better than those in the o3 system card, because the metrics in the o3 card reflected performance without browsing.” The gpt-oss model card ran it once more for the open-weight models on August 5, 2025, making the same point in general terms: browsing reduces hallucination because models “are able to look up information they do not have internal knowledge of.” The gpt-oss run is the benchmark’s last appearance in any card reviewed for this edition.

Days later, the GPT-5 system card arrived with its hallucination section intact — SimpleQA and other instruments still present — and PersonQA absent. To confirm the picture held, this publication saved and checked the full dated listing of OpenAI’s Deployment Safety Hub on August 24, 2026 — the census below.

In the eighteen cards published after August 5, 2025, the GPT-5 card among them, the benchmark appears exactly zero times. The company’s own April card had posed the question of the o3 regression; nothing in the set ever answered it.

Something did replace it, though not everywhere (several cards cover products with no factuality section at all), and not with the same question. The GPT-5 card, the first without PersonQA, grades factuality on prompts representative of real ChatGPT production conversations, scored by a model-based grader with web access. The current flagship-family cards evaluate factuality against user-flagged conversations; the GPT-5.5 and GPT-5.6 (Sol) cards put it verbatim — “de-identified ChatGPT conversations that users of our prior models have flagged as containing factual errors,” cases those cards describe as especially hallucination-prone, “not a representative slice of all production traffic.” Real measurement, arguably harder grading. But it measures what users complained about. Nothing in either method isolates the question PersonQA existed to isolate: whether the machine gets real, named people right.

What the record says, and what it doesn’t

That reading is real. What it licenses is narrower than it feels, so here are the boundaries, stated plainly: benchmarks get replaced all the time (they saturate, they leak into training data, better instruments supersede them). OpenAI never promised PersonQA was permanent, never explained its absence, and owes no one an explanation for routine engineering churn. Nothing in the public record supports any claim about why it stopped appearing, and this edition makes none. Nor did the benchmark vanish the moment it embarrassed anyone: it ran twice more, in other product lines, before it stopped. The April card itself even notes that comparison values from live models “may vary slightly from values published at launch” — o1’s own PersonQA number read 0.20 in February and 0.16 in April under the same label. The instrument was imperfect and the baseline unstable. The disappearance has no stated cause — and is unremarkable in kind.

The claim that survives every fair reading is harder to escape: the o3 regression has stood unexplained for sixteen months, and for the last twelve months (eighteen cards) the instrument that measured it has not appeared at all. The systems kept shipping, and the answers about people kept flowing. The gauge went dark.

The question that outlived the test

If your name, your license, your firm, or your record is an asset — and for anyone whose clients arrive by reputation, it is the asset — OpenAI’s current published system cards provide no person-specific benchmark result. Nothing in them tells a professional how reliably these systems represent real, named people. The only OpenAI benchmark in this review set designed specifically around questions and public facts about people has not appeared in a published card for more than a year — and the question its April reading raised is still open. Even a person-specific benchmark, measured in the aggregate, would not tell you whether the system gets your particular record right.

The reading has to come from your own record instead: what the machines actually return about you, checked against what is true, backed by identity infrastructure the machines can verify rather than guess at. OpenAI attributed the far lower browsing-enabled PersonQA rate in the ChatGPT Agent evaluation to the browsing configuration itself — and the cards do not identify which sources produced that difference or establish that browsing will improve every person-specific answer. That is the ground Answer Engine Authority™ occupies: making a professional’s public record clearer, more coherent, and easier to verify when an answer system consults external sources. It cannot guarantee what the system will retrieve, believe, or say. It can improve the evidence available to be found.

Run the one-line audit yourself: ask which benchmark in OpenAI’s current published system cards specifically measures accuracy about real, named people. Since August 5, 2025, across the twenty-eight documents in the census above, the answer is: none.