An Arbitrator Asked Two AI
Tools to Describe GPTZero

The award’s appendix says the arbitrator asked two AI tools “to each describe the GPTZero tool.” The award says he checked several citations. The award, as filed, does not discuss which GPTZero version the cited studies tested.

Within page 28 of an arbitration award dated Aug. 27, 2026, in a case over a teacher’s dismissal, after the words “My Query to Claude Opus 5,” a four-word question is printed in bold: What is “ChatGPT Zero”. Its answer says “ChatGPT Zero” is “a common (informal) name for GPTZero.” Two pages on, a second query, to GPT 5.4, runs three words: “What is GPTZero?” They are the questions the award prints from its author’s investigation of what GPTZero “does and whether it meets a standard of reliability.”

The award, signed by Arbitrator Thomas A. Kochan, is captioned as a grievance, “Unjust Termination,” between Concord Public Schools and Christine Apollos, a teacher whose evaluations were in dispute. The AI detector the award examines is GPTZero, which Ms. Apollos used and testified about. On this page, “as filed” means a document as this edition read it on Oct. 12, 2026. Neither the award nor her later court complaint, as filed, contains a GPTZero report.

The award states that the cautions in the National Academy of Arbitrators’ March 28, 2026 Advisory Memo “should apply with equal force when an arbitrator evaluates the weight, if any, to give to AI generated data” a party submits. A footnote (Advisory Memo 28, “The Arbitrator’s Use of AI”) discloses that he co-chairs the Academy’s AI Study Group, which drafted the memo; the Academy’s Board of Governors approved it. “I believe that, regardless of who introduces evidence based on an AI tool, it imposes an obligation on the arbitrator to assess whether the AI tool can be viewed as a credible source of accurate information,” the award states. That obligation, it adds, requires “an independent judgment of the reliability and/or potential bias of AI generated information.” After Ms. Apollos testified that she had not done anything to check the reliability of the tool or its outputs, the award records, the arbitrator conducted his own analysis with “two different, highly advanced, AI tools: Claude Opus 5 and ChatGPT 5.4.”

Two requests to describe

The appendix says he asked two AI tools “to each describe the GPTZero tool.” The appendix, as filed, does not report that he ran GPTZero on any document, or when the queries were run. The hearing transcript and the full chatbot answers are outside the sources we read.

The excerpt under the GPT 5.4 query states: “The detector cannot see context in which the text is situated. It only sees the text itself.” The answer printed under the Claude Opus 5 query states that “GPTZero itself concedes the key caveats,” including that results “should not be used to punish or as the final verdict.” GPTZero’s own FAQ, fetched Oct. 12, 2026, states that its results “should not be used to punish students.”

Independent testing, as printed

Under the line “Independent testing tells a messier story,” the answer printed under the Claude Opus 5 query shows a table with four columns, one headed “False positives on human text.” The column runs from 0.00% to 29%. The 0.00% is GPTZero’s row, labeled self-reported, 2026. The others are 29% for a “TwainGPT independent test,” 12.5% for TheHumanizeAI.pro, February 2026, and 1.3% for DecEptioner, February 2026. The sources this edition read do not establish whether those sites produced the printed figures. The printed answer says TwainGPT’s 29% figure “came from testing formal, structured human text, exactly the kind that GPTZero tends to struggle with,” crediting a source labeled “All About AI.”

The award’s summary, on page 22, states that the evidence indicated false-positive error rates for GPTZero “in the range of 10 to 20 percent, with one independent study finding substantially higher error rates.” It adds that the tools and their sources caution it “should not be used for decision making.” Neither “10 to 20” nor “decision making” appears in the printed appendix, which states that it copies “the relevant parts of the answers received.” The table’s figures are the printed answer’s, reproduced in the appendix, not the arbitrator’s own findings.

Three grounds, and a complaint

Ms. Apollos testified, in the award’s summary, that several of Principal Sparks’s summary evaluations were “likely written by AI.” In testimony, the award records, Mr. Sparks denied using any AI beyond the grammar checks built into the District’s word processing program. The award cites three grounds: the Academy’s cautionary memo, “this evidence” (on page 22, the evidence from his investigation) and her testimony that she took no actions to check the reliability of GPTZero’s estimates. It states: “I give no weight to the Grievant’s argument that AI wrote the Summative Evaluation or the summaries of any of the individual classroom observations.” Among the aspects of his testimony that, in the award’s words, “raised serious credibility concerns,” it lists his testifying that he did not use AI. Of the credibility issues of Mr. Sparks and Ms. Apollos, it states: “I do not judge them to be sufficient to disregard the overall testimony of either witness.” As filed, it makes no finding either way on whether he used AI.

The award concludes that “The District demonstrated just cause for dismissal” and that “The grievance is denied.” The award’s three stated grounds went to the AI-authorship argument; the award states the denial of the grievance in its Decision section, on page 26. Her complaint to vacate the award, stamped as filed in Suffolk Superior Court on Sept. 28, 2026, alleges the arbitrator “relied upon his own ex parte investigation into the credibility of an artificial intelligence tool, GPTZero.” The complaint alleges he never informed the parties he would conduct it and never gave them an opportunity to rebut its results. The award, as filed, neither says he gave the parties advance notice of his inquiry nor says he did not. The complaint alleges “misconduct” under M.G.L. c. 150C, § 11(a)(2) and, in a second count, that he exceeded his powers under § 11(a)(3) by placing the burden of proof on Ms. Apollos; it asks for a rehearing before new arbitrators.

Faithful to the sources, the award says

“I checked several of the citations and found these AI tools accurately reflected the underlying research pertaining to GPTZero,” the award states. The appendix describes a review of “some of the sources cited by these two advanced AI tools,” which, it states, the tools accurately reported. The award, as filed, does not say how many citations he checked, or which. We read the check it describes as one of fidelity: whether two chatbots’ summaries matched the sources they cited.

Fidelity answers one question. Fitness asks another: whether those studies concern GPTZero as it stood in August 2025, when, the complaint alleges, Ms. Apollos used it. A summary can be faithful to a study of something else. The award describes its analysis as “of the specific AI tool Ms. Apollos used.” The award, as filed, does not discuss which version the cited studies tested, or whether they concern the version she used. The complaint alleges the studies “analyze artificial intelligence tools that are significantly more advanced than those in existence in August 2025.” We make no judgment on which studies fit.

What the check can carry

The award states that the arbitrator should use AI generated results only if satisfied they “have been verified in some way,” and that “he or she owns responsibility for its use.” The award describes its check as a review of several citations and some of the sources the chatbots cited.

Where the argument stops

We make no claim that the arbitrator, the chatbots or any study was right or wrong, or about what decided the grievance. We make no finding that he breached the Academy’s Opinion No. 27, “Independent Investigation of Adjudicative Facts,” Opinion No. 28, “The Arbitrator’s Use of AI,” or any law. The sources this edition read on Oct. 12, 2026, contain no such finding by a court or Academy body. We could not read the trial-court docket that day.

We map the record, not any person, to Digital Derangement Syndrome™ (DDS™). On our reading, it is consistent with two patterns. One is Trust Transfer Failure (expertise never translated into machine-readable trust) in inverse: the opposite of DDS’s usual pattern of real expertise going unrecognized. The other is Contextual Ambiguity (a specific answer produced before the context determining its applicability had been established). The investigation’s evidence is one of the three grounds; neither printed query names a version, and the award does not address, as filed, whether the studies behind it concern the version she used. That mapping is editorial, not any party’s finding.