Anthropic Gives METR Deeper Access for Independent Evaluation

What happened

In an October 15 update, Anthropic said six METR staff were receiving employee-like access for frontier-model risk assessment. The described access includes internal tools and workspace, with exclusions including customer data and some sensitive company information.

Anthropic says METR can publish assessments without a company veto, subject to predefined redaction categories. The company retains authority over those categories, while METR can make certain statements about access received. Other evaluator relationships have different access and publication arrangements. The update describes a process; it does not report a completed independent finding that every model or deployment is safe.

Why access changes the evaluation question

Our interpretation is that an evaluator's ability to inspect a system matters alongside the tests it runs. A public model endpoint reveals behavior under selected prompts. Internal access may reveal relevant development decisions, incident handling or safeguard assumptions that a public trial cannot show.

The practical reading task is therefore to ask what evidence an assessment could see. Independence, access and publication freedom are separate properties. A recognizable evaluator name does not by itself establish all three, and a single published result should not be generalized beyond its stated scope.

What business buyers can learn

When a vendor supplies an evaluation report, record the model version, evaluation date, tested capabilities and excluded areas. Ask whether the assessor selected the tests independently and whether material negative results could be published. Keep unanswered questions visible in procurement notes.

Then run your own narrow acceptance test. A frontier-risk assessment and a customer-support accuracy test address different questions. Neither should be presented as a replacement for the other. A system can pass a broad capability assessment and still retrieve the wrong return policy in your application.

For a small pilot, assign an operator who did not write the prompts to review failures. Give that person access to the approved inputs and relevant outputs, with appropriate data restrictions. Preserve disagreements rather than averaging them into a reassuring score. A reviewer who cannot inspect failed cases cannot meaningfully explain the result.

What to watch next

Watch for METR's published assessments, the scope of actual access and explanations of material limitations. Distinguish the embedded arrangement from partnerships that still require approval to publish. Keep the original report date beside any claim reused in a later purchasing decision.

Published October 20 about the October 15 update. DiscoverAI has not audited the access arrangements or independently evaluated the models. For application-level testing, see test an AI workflow before changing models.