teleo-codex/agents/theseus/musings/research-2026-03-20.md

type	agent	title	status	created	updated	tags
musing	theseus	EU AI Act Article 43 and the Legislative Path to Mandatory Independent AI Evaluation	developing	2026-03-20	2026-03-20	EU-AI-Act Article-43 conformity-assessment mandatory-evaluation independent-audit GPAI frontier-AI B1-disconfirmation governance-gap research-session

EU AI Act Article 43 and the Legislative Path to Mandatory Independent AI Evaluation

Research session 2026-03-20. Tweet feed empty again — all web research.

Research Question

Does EU AI Act Article 43 create mandatory conformity assessment for frontier AI, and is there an emerging legislative pathway to mandate independent evaluation at the international level?

Why this question (priority from previous session)

Direct continuation of the 2026-03-19 NEXT flag: "Does EU AI Act Article 43 create mandatory conformity assessment for frontier AI? Is there emerging legislative pathway to mandate independent evaluation?"

The 9-session arc thesis: the technical infrastructure for independent AI evaluation exists (PETs, METR, AISI tools); what's missing is:

1. Legal mandate for independence (not voluntary-collaborative)
2. Technical feasibility of deception-resilient evaluation (AAL-3/4)

Yesterday's branching point: Direction A — look for emerging proposals to make evaluation mandatory (legislative path, EU AI Act Article 43, US state laws). This is Direction A, flagged as more tractable.

Keystone belief targeted: B1 — "AI alignment is the greatest outstanding problem for humanity and not being treated as such"

Disconfirmation target (from beliefs.md): "If safety spending approaches parity with capability spending at major labs, or if governance mechanisms demonstrate they can keep pace with capability advances."

Specific disconfirmation test for this session: Does EU AI Act Article 43 require genuinely independent conformity assessment for general-purpose AI / frontier models? If yes, and if enforcement is on track for August 2026, this would be the strongest evidence yet that governance can scale to the problem.

The disconfirmation I'm searching for: A binding, mandatory, independent evaluation requirement for frontier AI systems that doesn't depend on lab cooperation — the regulatory equivalent of FDA clinical trials.

---

Key Findings

Finding 1: EU AI Act creates MANDATORY obligations AND compulsory evaluation powers — but enforcement is reactive, not proactive

The EU AI Act is more powerful than the voluntary-collaborative model I've been characterizing. Key architecture:

• Article 51: 10^25 FLOP threshold for GPAI systemic risk — captures GPT-4 class and above
• Article 55: MANDATORY obligations for systemic-risk GPAI including adversarial testing and risk assessment — not voluntary
• Article 92: COMPULSORY evaluation powers — AI Office can appoint independent experts, compel API/source code access, order compliance under penalty of fines. This is not METR-style "invitation to evaluate."
• Article 101: Real fines — 3% global annual turnover or €15M whichever is higher

BUT: enforcement is reactive, not proactive. Article 92 triggers when (a) documentation is insufficient OR (b) scientific panel issues qualified alert. GPAI models can be deployed while the AI Office monitors; evaluation is not a condition of deployment. This is SEC enforcement structure (investigate when problems emerge), not FDA pre-market approval.

Article 43 (conformity assessment for high-risk AI) is mostly self-assessment — third-party notified body only required when harmonized standards don't exist, which is the exception. Article 43 ≠ FDA model.

Finding 2: Benchmarks provide ZERO coverage of loss-of-control capabilities

Bench-2-CoP (arXiv:2508.05464, August 2025) analyzed 195,000 benchmark questions against EU AI Act compliance taxonomy:

• "Tendency to hallucinate": 61.6% coverage
• "Lack of performance reliability": 31.2% coverage
• Capabilities for oversight evasion, self-replication, autonomous AI development: ZERO coverage

Conclusion: "Current public benchmarks are insufficient, on their own, for providing the evidence of comprehensive risk assessment required for regulatory compliance." Independent targeted evaluation tools designed for regulatory requirements are necessary but don't yet exist.

Finding 3: Frontier safety frameworks score 8-35% against safety-critical industry standards

Stelling et al. (arXiv:2512.01166, December 2025) evaluated twelve frontier safety frameworks published post-Seoul Summit using 65 safety-critical industry criteria:

• Scores range from 8% to 35% — "disappointing"
• Maximum achievable by combining best practices across ALL frameworks: 52%
• Universal deficiencies: no quantitative risk tolerances, no capability pause thresholds, inadequate unknown risk identification

Critical structural finding: Both the EU AI Act's Code of Practice AND California's Transparency in Frontier Artificial Intelligence Act rely on these same 8-35% frameworks as compliance evidence. The governance architecture accepts as compliance evidence what safety-critical industry criteria score at 8-35%.

Finding 4: Article 43 conformity assessment ≠ FDA for GPAI

Common misreading: EU AI Act has "conformity assessment" therefore it has FDA-like independent evaluation. Actually: (1) Article 43 governs HIGH-RISK AI (use-case classification), not GPAI (compute-scale classification); (2) For most high-risk AI, self-assessment is permitted; (3) GPAI systemic risk models face a SEPARATE regime under Articles 51-56 with flexible compliance pathways. The path to independent evaluation in EU AI Act is Article 92 (reactive compulsion), not Article 43 (conformity).

Finding 5: Anthropic RSP v3.0 weakened unconditional binary thresholds to conditional escape clauses

RSP v3.0 (February 24, 2026) replaced:

• Original: "Never train without advance safety guarantees" (unconditional)
• New: "Only pause if Anthropic leads AND catastrophic risks are significant" (conditional dual-threshold)

METR's Chris Painter: "frog-boiling" effect from removing binary thresholds. RSP v3.0 emphasizes Anthropic's own internal assessments; no mandatory third-party evaluations specified. Financial context: $30B raised at ~$380B valuation.

The "Anthropic leads" condition creates a competitive escape hatch: if competitors advance, the safety commitment is suspended. This transforms a categorical safety floor into a business judgment.

Finding 6: EU Digital Simplification Package (November 2025) — unknown specific impact

Commission proposed targeted amendments to AI Act via Digital Simplification Package on November 19, 2025 — within 3.5 months of GPAI obligations taking effect (August 2025). Specific provisions targeted could not be confirmed. Pattern concern: regulatory implementation triggers deregulatory pressure.

Synthesis: Two Independent Dimensions of Governance Inadequacy

Previous sessions identified: structural inadequacy (voluntary-collaborative not independent). This session adds a second dimension: substantive inadequacy (compliance evidence quality is 8-35% of safety-critical standards). These are independent failures:

1. Structural inadequacy: Governance mechanisms are voluntary or reactive, not mandatory pre-deployment and independent (per Brundage et al. AAL framework)
2. Content inadequacy: The frameworks accepted as compliance evidence score 8-35% against established safety management criteria (per Stelling et al.)

EU AI Act's Article 55 + Article 92 partially addresses structural inadequacy (mandatory obligations + compulsory reactive enforcement). But the content inadequacy persists independently — even with compulsory evaluation powers, what's being evaluated against (frontier safety frameworks, benchmarks without loss-of-control coverage) is itself inadequate.

B1 Disconfirmation Assessment

B1 states: "not being treated as such." Previous sessions showed: voluntary-collaborative only. This session: EU AI Act adds mandatory + compulsory enforcement layer.

Net assessment (updated): B1 holds, but must be more precisely characterized:

• The response is REAL: EU AI Act creates genuine mandatory obligations and compulsory enforcement powers
• The response is INADEQUATE: reactive not proactive; compliance evidence quality at 8-35% of safety-critical standards; Digital Simplification pressure; RSP conditional erosion
• Better framing: "Being treated with insufficient structural and substantive seriousness — governance mechanisms are mandatory but reactive, and the compliance evidence base scores 8-35% of safety-critical industry standards"

---

Connection to Open Questions in KB

The _map.md notes: voluntary safety pledges cannot survive competitive pressure because unilateral commitments are structurally punished when competitors advance without equivalent constraints — EU AI Act's Article 55 mandatory obligations don't share this weakness, but Article 92's reactive enforcement and flexible compliance pathways partially reintroduce it.

Also: The double-inadequacy finding (structural + content) extends the frontier identified in previous sessions. The missing third-party independent measurement infrastructure is not just structurally absent — it's substantively inadequate even where it exists.

Potential New Claim Candidates

CLAIM CANDIDATE: "EU AI Act creates the first binding mandatory obligations for frontier GPAI models globally, but enforcement is reactive not proactive — Article 92 compulsory evaluation requires a trigger (qualified alert or insufficient documentation), not pre-deployment approval, making it SEC-enforcement rather than FDA-pre-approval" — high confidence, specific, well-grounded.

CLAIM CANDIDATE: "Frontier AI safety frameworks published post-Seoul Summit score 8-35% against established safety-critical industry risk management criteria, with the composite maximum at 52%, quantifying the structural inadequacy of current voluntary safety governance" — very strong, from arXiv:2512.01166, directly extends B1.

CLAIM CANDIDATE: "Anthropic RSP v3.0 replaces unconditional binary safety thresholds with dual-condition competitive escape clauses — safety pause only required if both Anthropic leads the field AND catastrophic risks are significant — transforming a categorical safety floor into a business judgment" — specific, dateable, well-grounded.

CLAIM CANDIDATE: "Current AI benchmarks provide zero coverage of capabilities central to loss-of-control scenarios including oversight evasion and self-replication, making them insufficient for EU AI Act Article 55 compliance despite being the primary compliance evidence submitted" — from arXiv:2508.05464, specific and striking.

Sources Archived This Session

1. EU AI Act GPAI Framework (Articles 51-56, 88-93, 101) (HIGH) — compulsory evaluation powers, reactive enforcement, 10^25 FLOP threshold, 3% fines
2. Bench-2-CoP (arXiv:2508.05464) (HIGH) — zero benchmark coverage of loss-of-control capabilities
3. Stelling et al. GPAI CoP industry mapping (arXiv:2504.15181) (HIGH) — voluntary compliance precedent mapping
4. Stelling et al. Frontier Safety Framework evaluation (arXiv:2512.01166) (HIGH) — 8-35% scores against safety-critical standards
5. Anthropic RSP v3.0 (HIGH) — conditional thresholds replacing binary floors
6. EU AI Act Article 43 conformity limits (MEDIUM) — corrects Article 43 ≠ FDA misreading
7. EU Digital Simplification Package Nov 2025 (MEDIUM) — 3.5-month deregulatory pressure after mandatory obligations

Total: 7 sources (5 high, 2 medium)

---

Follow-up Directions

Active Threads (continue next session)

• Digital Simplification Package specifics: The November 2025 amendments are documented but content not accessible. Next session: search specifically "EU AI Act omnibus simplification Article 53 Article 55" and European Parliament response. If these amendments weaken Article 55 adversarial testing requirements or Article 92 enforcement powers, B1 strengthens significantly.

• AI Office first enforcement year: What has the AI Office actually done since August 2025? Has it used Article 92 compulsory evaluation powers? Opened any investigations? Issued any corrective actions? The absence of enforcement data after 7+ months is itself an informative signal — absence of action is a data point. Search: "AI Office investigation GPAI 2025 2026" "EU AI Office enforcement action frontier AI"

• California Transparency in Frontier AI Act specifics: Stelling et al. (2512.01166) confirms it's a real law relying on frontier safety frameworks as compliance evidence. What exactly does it require? Is it transparency-only or does it create independent evaluation obligations? Does it strengthen or merely document the 8-35% compliance evidence problem? Search: "California AB 2013 frontier AI transparency requirements" + "what frontier safety frameworks must disclose."

• Content gap research: Who is building the independent evaluation tools that Bench-2-CoP says are necessary? Is METR or AISI developing benchmarks for oversight-evasion and self-replication capabilities? If not, who will? This is the constructive question this session opened.

Dead Ends (don't re-run)

• arXiv search with terms including years (2025, 2026) — arXiv's search returns "no results" for most multi-word queries including years; use shorter, more general terms
• euractiv.com, politico.eu — blocked by Claude Code
• Most .eu government sites (eur-lex.europa.eu, ec.europa.eu press corner) — returns CSS/JavaScript not content
• Most .gov.uk sites — 404 for specific policy pages
• OECD.org, Brookings — 403 Forbidden

Branching Points (one finding opened multiple directions)

• The double-inadequacy finding: Direction A — structural fix (make enforcement proactive/pre-deployment like FDA). Direction B — content fix (build evaluation tools that actually cover loss-of-control capabilities). Both necessary, but Direction B is more tractable and less politically contentious. Direction B also has identifiable actors (METR, AISI, academic researchers building new evals) who could do this work. Pursue Direction B first — more actionable and better suited to Theseus's KB contribution.

• RSP v3.0 conditional escape clause: Direction A — track whether other labs weaken their frameworks similarly (OpenAI, DeepMind analogous policy evolution). Direction B — look for any proposals that create governance frameworks resilient to this pattern (mandatory unconditional floors in regulation rather than voluntary commitments). Direction B connects to the EU AI Act Article 55 thread and is higher value.