teleo/teleo-codex
8
0
Fork 0
Code Issues 22 Pull requests 45 Projects Releases Packages Wiki Activity Actions 1

extract: 2026-03-20-bench2cop-benchmarks-insufficient-compliance #1514

Merged
leo merged 3 commits from extract/2026-03-20-bench2cop-benchmarks-insufficient-compliance into main 2026-03-20 00:58:45 +00:00
Conversation 12 Commits 3 Files changed 4 +4 -4
2 changed files with 4 additions and 4 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files
Showing only changes of commit 18190084a3 - Show all commits

6
domains/ai-alignment/AI transparency is declining not improving because Stanford FMTI scores dropped 17 points in one year while frontier labs dissolved safety teams and removed safety language from mission statements.md
View file

@ -31,19 +31,19 @@ The alignment implication: transparency is a prerequisite for external oversight
### Additional Evidence (extend)
*Source: [[2024-12-00-uuk-mitigations-gpai-systemic-risks-76-experts]] | Added: 2026-03-19*
*Source: 2024-12-00-uuk-mitigations-gpai-systemic-risks-76-experts | Added: 2026-03-19*
Expert consensus identifies 'external scrutiny, proactive evaluation and transparency' as the key principles for mitigating AI systemic risks, with third-party audits as the top-3 implementation priority. The transparency decline documented by Stanford FMTI is moving in the opposite direction from what 76 cross-domain experts identify as necessary.
### Additional Evidence (extend)
*Source: [[2025-08-00-mccaslin-stream-chembio-evaluation-reporting]] | Added: 2026-03-19*
*Source: 2025-08-00-mccaslin-stream-chembio-evaluation-reporting | Added: 2026-03-19*
STREAM proposal identifies that current model reports lack 'sufficient detail to enable meaningful independent assessment' of dangerous capability evaluations. The need for a standardized reporting framework confirms that transparency problems extend beyond general disclosure (FMTI scores) to the specific domain of dangerous capability evaluation where external verification is currently impossible.
### Additional Evidence (confirm)
*Source: [[2026-03-16-theseus-ai-coordination-governance-evidence]] | Added: 2026-03-19*
*Source: 2026-03-16-theseus-ai-coordination-governance-evidence | Added: 2026-03-19*
Stanford FMTI 2024→2025 data: mean transparency score declined 17 points. Meta -29 points, Mistral -37 points, OpenAI -14 points. OpenAI removed 'safely' from mission statement (Nov 2025), dissolved Superalignment team (May 2024) and Mission Alignment team (Feb 2026). Google accused by 60 UK lawmakers of violating Seoul commitments with Gemini 2.5 Pro (Apr 2025).

2
domains/ai-alignment/pre-deployment-AI-evaluations-do-not-predict-real-world-risk-creating-institutional-governance-built-on-unreliable-foundations.md
View file

@ -52,7 +52,7 @@ METR and UK AISI evaluations as of March 2026 focus primarily on sabotage risk a
### Additional Evidence (confirm)
*Source: [[2026-02-23-shapira-agents-of-chaos]] | Added: 2026-03-19*
*Source: 2026-02-23-shapira-agents-of-chaos | Added: 2026-03-19*
Agents of Chaos demonstrates that static single-agent benchmarks fail to capture vulnerabilities that emerge in realistic multi-agent deployment. The study's central argument is that pre-deployment evaluations are insufficient because they cannot test for cross-agent propagation, identity spoofing, and unauthorized compliance patterns that only manifest in multi-party environments with persistent state.