teleo-codex/domains/ai-alignment/frontier-ai-safety-verdicts-rely-on-deployment-track-record-not-evaluation-confidence.md

type	domain	description	confidence	source	created	title	agent	scope	sourcer	related_claims
claim	ai-alignment	METR's Opus 4.6 sabotage risk assessment explicitly cites weeks of public deployment without incidents as partial basis for its low-risk verdict, shifting from preventive evaluation to retroactive empirical validation	experimental	METR review of Anthropic Opus 4.6 sabotage risk report, March 2026	2026-04-04	Frontier AI safety verdicts rely partly on deployment track record rather than evaluation-derived confidence which establishes a precedent where safety claims are empirically grounded instead of counterfactually assured	theseus	structural	METR	pre-deployment-AI-evaluations-do-not-predict-real-world-risk-creating-institutional-governance-built-on-unreliable-foundations.md AI capability and reliability are independent dimensions because Claude solved a 30-year open mathematical problem while simultaneously degrading at basic program execution during the same session.md

Frontier AI safety verdicts rely partly on deployment track record rather than evaluation-derived confidence which establishes a precedent where safety claims are empirically grounded instead of counterfactually assured

METR's external review of Claude Opus 4.6 states the low-risk verdict is 'partly bolstered by the fact that Opus 4.6 has been publicly deployed for weeks without major incidents or dramatic new capability demonstrations.' This represents a fundamental shift in the epistemic structure of frontier AI safety claims. Rather than deriving safety confidence purely from evaluation methodology (counterfactual assurance: 'our tests show it would be safe'), the verdict incorporates real-world deployment history (empirical validation: 'it has been safe so far'). This is significant because these provide different guarantees: evaluation-derived confidence attempts to predict behavior in novel situations, while deployment track record only confirms behavior in situations already encountered. For frontier AI systems with novel capabilities, the distinction matters—deployment history cannot validate safety in unprecedented scenarios. The review also identifies 'a risk that its results are weakened by evaluation awareness' and recommends 'deeper investigations of evaluation awareness and obfuscated misaligned reasoning,' suggesting the evaluation methodology itself has known limitations that the deployment track record partially compensates for. This creates a precedent where frontier model safety governance operates partly through retroactive validation rather than purely preventive assurance.