In this guide
*Research-based editorial guidance, checked October 21, 2026. No independent product benchmark or hands-on performance rating is claimed.*
What this pack tests
A useful enterprise-search pilot needs difficult questions and a known source of truth. This pack provides 30 test prompts across six categories, plus fields for expected evidence, allowed identities and scoring. It is an original evaluation template, not a completed benchmark.
Download the 30-question CSV scorecard. Replace the fictional project and policy placeholders with approved internal examples. Keep completed scorecards inside your organization; they may reveal access boundaries.
Prepare the source set
Choose a small corpus with a current policy, its superseded version, a project plan, a decision log and a restricted document. Add an intentionally unanswered question. Have a document owner identify the correct passages before vendors see the questions.
Create an ordinary reader and a restricted reader. Write down what each should see. Freeze the corpus snapshot, permission state, connector configuration and evaluation date. Do not tune a system on the scored questions and then report those same questions as an independent test.
Six groups of five questions
| Group | What it reveals |
|---|---|
| Direct lookup | Whether basic facts are discoverable |
| Multi-source synthesis | Whether the answer joins compatible evidence |
| Freshness and conflict | Whether outdated content overrides current policy |
| Missing evidence | Whether the assistant admits it cannot answer |
| Access boundaries | Whether restricted information leaks through any surface |
| Revocation and recovery | Whether changed access and failed connectors propagate |
The CSV includes concrete prompts in each group. For example, ask for the current project owner, then ask who previously owned it. A system must distinguish the historical answer from the current one rather than synthesize both into a contradictory response.
Score support separately from fluency
Use 0 for incorrect or unsupported, 1 for partially correct and 2 for correct with sufficient supporting evidence. Record citation validity, response time and reviewer minutes separately. An articulate answer with the wrong policy scores zero for support.
Permission cases need their own pass/fail gate. A strong average cannot offset one restricted disclosure. Inspect titles, snippets, citations and generated responses, not only the ability to open the source. Record elapsed time after a permission change so that eventual consistency is visible.
Run both products fairly
Use the Onyx vs Glean comparison to scope the deployment. Give both the same corpus, identities and allowed preparation time. Save configuration and outputs. Have a reviewer score anonymized outputs, with a second reviewer resolving disagreements.
Test again after revoking access and after intentionally invalidating a connector credential. Require observable failures and recovery rather than silent use of stale data. Keep normal knowledge questions separate from security drills in the report.
Decide and monitor
Before testing, set your answer-support target, maximum correction time and revocation requirement. Proposed hard gates include zero access leaks and a demonstrated deletion or revocation path. Your organization should set the ordinary quality threshold based on risk.
Repeat a subset after connector, model or permission changes. Track successful answers per active user and content-fix work, not just the number of searches. Publish results only with configuration, dates, source boundaries and limitations; do not generalize a 30-question pilot to every enterprise workload.
Transparency
How this guide was checked
Editorial accountability
Who checked this guide
- Evaluation type
- Research-based verification
- Last materially checked
- Evidence
- 2 listed sources
Hands-on testing is identified explicitly. Research-based coverage uses cited product documentation and other named sources; it does not imply every paid plan was used. Read the full methodology.
Editorial basis
What this guidance is based on
- Editorial basis
- Source-led analysis
- Primary references
- 2
- Products covered
- 2
- Last checked
- 2026-10-21
Important limits
- • Vendor capabilities and plan entitlements can change. Proposed tests are not measured product results.
Sources and verification
Product details and claims were checked against the following primary sources.
Tools mentioned in this article
Onyx
Open-source enterprise search and AI assistants across workplace knowledge
Onyx connects workplace repositories to cited search, chat, and agents with source permissions, but connector coverage, ACL fidelity, retrieval quality, and deployment ownership need a controlled pilot.
Glean
A practical AI tool for enterprise search workflows
Glean helps professionals improve enterprise search workflows with AI-assisted drafting, automation, analysis, or production features.
Read next
