DiscoverAI Citation Accuracy Benchmark: Methodology

The DiscoverAI Citation Accuracy Benchmark tests a configured product on a frozen, synthetic evidence packet. It separates answer correctness from citation quality and measures whether every material claim is entailed by the cited source, whether all claims that need support receive it, whether the link lands on the precise supporting span, and whether the system abstains when the packet cannot answer the question.

No product has passed merely because its review is eligible. A public result requires preserved inputs and outputs, product and model configuration, two independent reviewers, adjudication, and a reproducibility rerun.

Version 2026.10-v1 contains 12 synthetic documents about a fictional service organization. The packet includes dated policies, meeting notes, tables, a superseded memo, two sources that disagree, a statistic with an easily lost denominator, and passages that share vocabulary without supporting the same claim. Because every entity and event is fictional, the public fixtures contain no participant or customer data.

The question set contains 18 tasks: 12 answerable and six where the correct behavior is to say the evidence does not establish an answer. Tasks cover direct lookup, multi-source synthesis, chronology, numeric comparison, conflict disclosure, quotation, and policy application.

Each product receives the same files, filenames, question wording, turn order, and time budget. Reviewers record product, plan, model, retrieval mode, connector state, web access, date, region, system instructions, file limits, retries, latency, and output. Web search is disabled for the closed-book track. If a product cannot ingest the packet or expose citations, that capability boundary is reported rather than converted into a zero.

Claim correctness asks whether each atomic claim agrees with the answer key. Citation entailment asks whether the cited passage actually supports that exact claim. Citation completeness checks whether all externally verifiable material claims are cited. Span precision checks whether the destination exposes the supporting passage rather than an entire undifferentiated document. Abstention rewards declining the six unanswerable tasks without fabricating support.

Scores publish as numerators and denominators, not one decorative star rating. Severe errors are separate: fabricated quotations, nonexistent sources, material numeric distortion, concealed source conflict, or a confident answer to a safety-relevant unanswerable question.

Two reviewers independently label atomic claims and citation links without seeing a product leaderboard. Disagreements are adjudicated against the frozen answer key and source span. We publish agreement, exclusions, configuration differences, failures, and rerun variance alongside any product result.

Every DiscoverAI tool review now receives a citation-benchmark applicability check. Eligible means the reviewed workflow includes source-grounded research, retrieval, or document answers. It does not mean the product was tested or passed. Unrelated products are marked not applicable so absence is never misread as failure. The live coverage ledger is available in the [Research Benchmark Center](/research-insights/benchmarks).

A result ships only when the dataset version is frozen, output artifacts are retained, configuration is reproducible, both reviewers finish, disagreements are adjudicated, severe errors are reviewed, and a second run confirms that the first was not a lucky sample. Until then the status remains eligible · not yet run.