In an AI-Q Study, 84.7% of
Identified Final-Report Errors
Traced to the Orchestrator.

A peer-reviewed error-localization study, accepted to EMNLP 2026, traced where citation and faithfulness errors enter multi-agent research systems, including the evaluated configuration of NVIDIA’s AI-Q, an open reference architecture positioned to “deliver trusted business insights.” Researchers attributed 84.7% of the identified final-report errors to the orchestrator: the layer that assembles the cited report from researcher notes, planner output, raw search-engine snippets, and a source registry, without inspecting the full source documents. The components that summarized one document at a time made the fewest mistakes.

By William Douglas Baker

Founding Editor

The question sounds like an engineering detail until your name is in the report: when a machine-written research brief gets something wrong, which part of the machine made the error? The wrong can be small: a claim pinned to a source that never made it, a citation pointing at work that says otherwise. A research team at Bar-Ilan University, UNC Chapel Hill, and the University of Texas at Austin built a method to trace where those errors entered the pipeline, following identified citation and faithfulness errors in a finished report back to the specific agent that introduced them. The paper, accepted to the main conference of EMNLP 2026, examined three top-ranked open-source “deep research” systems, the multi-agent pipelines that search, read, and assemble long-form cited reports. Among them: AI-Q, the open reference architecture whose NVIDIA repository description promises to “deliver trusted business insights.”

The finding arrives in the paper’s own bolded words: “The majority of mistakes start at the orchestrator.” In the evaluated AI-Q configuration, the researchers found “84.7% of final-report errors in AI-Q originate at the orchestrator, roughly 31% of them hallucinations” — the rest citation mistakes. The other two systems ran 52.6% and 100%: a three-system observation, not a controlled architectural effect, and the authors caution that the capability of the underlying model appeared to explain the dominant error type better than the shared multi-agent architecture. The authors’ image for why is a children’s game: information moves through the pipeline’s agents like a “telephone game.” In their words: “As content crosses agent boundaries, and often gets compressed, both core information as well as citations may get corrupted.”

Be precise about what the orchestrator is, because the precision is the story. In the paper’s evaluated build, the orchestrator is the main agent — the one that assembles the final, cited report. And it is not blind. Per the study’s appendices, the evaluated orchestrator received the researcher agents’ synthesized notes with raw search-engine snippets of the cited URLs appended, along with planner output and a source registry: a map of citation identifiers to URLs. Notes, snippets, and a list of where the sources live. What it does not inspect is the full source document. One taxonomy caution belongs on the record: NVIDIA’s current product page slices the pipeline with its own newer names — an intent-classifying orchestration node at the front door, a separate writer stage for final synthesis — so the paper’s finding attaches to the assembly layer of the build the researchers evaluated, whatever label the box carries this quarter. The paper’s own summary line names the one exception in the error pattern: most agents “make a lot of mistakes, except those that summarize a single document.” The layer that assembles the final, cited account works from mediated evidence — and it is the layer to which, in the evaluated system, most of the identified final-report errors were attributed.

The Telephone Game, Measured

What This Study Does Not Say

It does not say AI-Q is broken. The system placed among the top-ranked open architectures the study examined, and it is an open reference. Publishing its structure for inspection is exactly what an open reference is for; the study reads that published structure, which is analysis, not indictment. It does not say 84.7% of reports are wrong; the figure is the share of identified errors attributed to one layer, in one configuration, on 20 benchmark queries. The paper states the tension itself: that share lands at the orchestrator despite its having one of the lowest agent-level mistake rates in its own system, 30.9% against 70.8% at its least reliable researcher, “which illustrates the value of system-level evaluation on top of agent-level evaluation.” Where identified errors enter a report and how often a component errs per sentence are different measurements. And it does not measure harm to any named professional — no reputation was tracked, no client was lost on camera. The study localizes error and stops there. What can be said on its evidence is narrower and colder: in the evaluated AI-Q configuration, the component to which most identified final-report errors were attributed assembled its account from mediated evidence without inspecting the full documents.

Through the Agentics lens — and in this record’s clinical register, rather than the researchers’ diagnosis — the mechanism has a name: Signal Fragmentation, one of the five clinical characteristics of Digital Derangement Syndrome™. The signal does not disappear in these pipelines: it fragments in transit. What a source actually says survives each hand-off as notes, snippets, and identifiers — and the final account is assembled from that mediated evidence, never from the full documents. Trust Transfer Failure, a second of the five characteristics, rides along: the product promise on the label — trusted business insights — is underwritten upstream at the assembly layer. That is the component to which most identified final-report errors were attributed.

Edition No. 061 filed the litigated version of this distance on September 2nd: records submitted for review, a complaint alleging the review never happened. Today’s edition files the measured version, inside the machine itself. Presence in the pipeline is not full-document inspection — and this time there is a number on the distance. For Answer Engine Authority™, the design question extends beyond retrieval: how much of the source’s meaning survives the hand-off? This study did not test AEA™. The study’s own repairs still point the direction: change what the assembly layer holds — raw source snippets in place of summaries — and citation quality moves. What the assembly layer holds is the whole game.

The paper set out to assign blame inside a machine, and did. What it leaves on the record reaches past the pipeline it measured: the account of you that reaches a reader is written by whatever the assembly layer holds, and the assembly layer holds what the relay gave it. For the person named in the report, the evidence available to the assembly layer remains invisible. This study measured the relay once — and showed where its identified errors entered.