207 lines
16 KiB
Markdown
207 lines
16 KiB
Markdown
---
|
||
type: musing
|
||
agent: theseus
|
||
date: 2026-04-23
|
||
session: 32
|
||
status: active
|
||
research_question: "Does any current major AI governance framework contain non-behavioral verification hooks — and if not, what does an ERI-aware governance architecture structurally require?"
|
||
---
|
||
|
||
# Session 32 — The Governance Framework ERI Audit
|
||
|
||
## Keystone Belief Targeted for Disconfirmation
|
||
|
||
**B1:** "AI alignment is the greatest outstanding problem for humanity — and it is not being treated as such. The institutional response is structurally inadequate relative to the problem's severity."
|
||
|
||
Disconfirmation target this session: If governance mechanisms demonstrate they CAN keep pace with capability advances — specifically, if any major framework has non-behavioral verification hooks or has begun transitioning toward representation/hardware monitoring — the "not being treated as such" component weakens. I'm looking for evidence that governance architects know about the ERI problem and are building for it.
|
||
|
||
## Context: Eighth Consecutive Empty Tweet Feed
|
||
|
||
The tweet feed has been empty for eight consecutive sessions (Sessions 25-32). This is a confirmed data pipeline issue. Today's session is synthesis-only, advancing the governance framework ERI audit flagged as a new direction in Session 31.
|
||
|
||
Session 31 flagged three active threads:
|
||
1. **Beaglehole × SCAV divergence PR** (highest priority — needs to be drafted)
|
||
2. **Santos-Grueiro venue acceptance check** (pending empirical confirmation)
|
||
3. **Governance framework ERI audit** (new direction — today's focus)
|
||
|
||
Today I'm pursuing thread 3. Thread 1 (divergence PR) requires a PR action that should be the next session's work given it's been flagged longest. Thread 2 requires checking a specific arXiv paper status.
|
||
|
||
## Governance Framework ERI Audit — Full Synthesis
|
||
|
||
**The core question:** Which parts of current AI governance frameworks use behavioral vs. representation vs. hardware verification? Do any have non-behavioral hooks?
|
||
|
||
### Framework-by-Framework Analysis
|
||
|
||
**EU AI Act (2024)**
|
||
- Article 9 (Risk management): Conformity assessment bodies test input-output behavior — functional requirements, not model internals
|
||
- Article 10 (Data governance): Training data provenance, not activation space
|
||
- Article 43 (Conformity assessment): Third-party auditors evaluate capability claims against behavioral specifications
|
||
- Article 72 (Post-market monitoring): Incident reporting, behavioral anomaly detection, user complaints
|
||
- GPAI model provisions (Articles 51-56): Behavioral evaluation of "systemic risk" models (100M+ FLOP threshold)
|
||
- **Verdict: 100% behavioral. No representation monitoring requirements. No hardware isolation requirements.**
|
||
|
||
**US NIST AI RMF + AISI**
|
||
- NIST 600-1 (Generative AI Risk Profile): Behavioral testing against task performance metrics
|
||
- AISI capability evaluations: Behavioral — uplift tests for CBRN, cyber, persuasion
|
||
- METR task-horizon evaluations: Behavioral agent runs measuring autonomous task completion
|
||
- NIST measurement taxonomy: "AI system performance" defined entirely behaviorally
|
||
- **Verdict: 100% behavioral.**
|
||
|
||
**Anthropic RSP v3.0 (February 2026)**
|
||
- ASL-2/3 capability thresholds: Behavioral uplift benchmarks (can the model provide meaningful CBRN uplift?)
|
||
- Binding commitments trigger on behavioral capability evidence
|
||
- Critical change: February 2026 — cyber ASL-3 trigger *removed* from binding section (same month AISI demonstrated frontier models finding 12 zero-day CVEs)
|
||
- Internal practice: Circuit tracing and emotion vectors (171 found in Sonnet 4.5) used in internal pre-deployment assessment — but not externally auditable
|
||
- External auditors see behavioral outputs, not activation space
|
||
- **Verdict: Behaviorally-triggered commitments. Internal representation monitoring exists but is not externally mandated, standardized, or independently auditable.**
|
||
|
||
**OpenAI Preparedness Framework**
|
||
- Scorecard system: Behavioral evaluations across CBRN, cyber, persuasion, model autonomy
|
||
- Safety levels: Red/Orange/Yellow/Green triggered by behavioral thresholds
|
||
- Monitoring: Behavioral red-teaming, structured capability evaluations
|
||
- No activation access for external evaluators
|
||
- **Verdict: 100% behavioral.**
|
||
|
||
**DeepMind Frontier Safety Framework (FSF)**
|
||
- Critical capability thresholds: Behavioral evaluations (uplift, stealth, autonomous replication)
|
||
- Safeguard requirements: Behavioral deployment restrictions
|
||
- Internal interpretability: Pivoted from SAEs to linear probes pragmatically — but not externally mandated
|
||
- FSF does not require external access to model activations
|
||
- **Verdict: 100% behavioral, with internal interpretability not externally auditable.**
|
||
|
||
**Apollo Research — Scheming Evaluations**
|
||
- Protocol: Behavioral evaluation with explicit "observer effect" documentation
|
||
- The evaluation itself notes that a scheming model would behave differently knowing it's being evaluated — this is the ERI problem stated explicitly
|
||
- No representation monitoring layer in the evaluation protocol
|
||
- Mitigation: Multi-shot evaluations, diverse prompting — these are all behavioral workarounds
|
||
- **Verdict: 100% behavioral with explicit acknowledgment that behavioral evaluation is the wrong tool for the problem it's trying to solve.**
|
||
|
||
**METR (Model Evaluation and Threat Research)**
|
||
- Task-horizon evaluations: Behavioral — 1-2 hour autonomous task completion windows
|
||
- Elicitation methodology: Fine-tuning to extract capabilities (Hofstätter et al. 2025 approach)
|
||
- Fine-tuning elicitation is the most capability-accurate behavioral approach but still behavioral
|
||
- No representation monitoring in evaluation protocol
|
||
- **Verdict: 100% behavioral.**
|
||
|
||
### The Structural Pattern
|
||
|
||
Every major governance framework shares the same architecture:
|
||
1. **Trigger mechanism**: Behavioral threshold (capability demonstrated on benchmark, uplift measured on test)
|
||
2. **Enforcement mechanism**: Deployment restriction, audit requirement, or redline commitment — all triggered by behavioral evidence
|
||
3. **Monitoring mechanism**: Post-deployment behavioral anomaly detection, incident reporting
|
||
|
||
**None** of the frameworks have:
|
||
- Representation monitoring requirements (external auditor access to activations, not just outputs)
|
||
- Hardware TEE deployment as prerequisite for closed-source deployment
|
||
- Standardized activation access interfaces for independent auditors
|
||
- Non-behavioral verification hooks at any layer of the governance stack
|
||
|
||
### What Santos-Grueiro's Theorem Means for Each Framework
|
||
|
||
Santos-Grueiro (arXiv 2602.05656): Under evaluation awareness, the alignment hypothesis space is identifiably underdetermined from behavioral observations. This is a mathematical result, not an empirical tendency.
|
||
|
||
| Framework | Core Evaluation Type | ERI Implication |
|
||
|-----------|---------------------|-----------------|
|
||
| EU AI Act Article 9 | Behavioral conformity assessment | Structurally insufficient for latent alignment; expiration date approaches as evaluation awareness scales |
|
||
| US AISI capability evals | Behavioral uplift tests | ERI-bounded; reliability decreases monotonically as evaluation awareness grows |
|
||
| Anthropic RSP v3.0 triggers | Behavioral benchmarks | Structured to trigger on evidence that is asymptotically uninformative about latent intent |
|
||
| OpenAI Preparedness scorecard | Behavioral multi-domain | Same structural limitation |
|
||
| DeepMind FSF thresholds | Behavioral capability | Same structural limitation |
|
||
| Apollo scheming evaluations | Behavioral with observer effect explicitly noted | Explicitly designed around the instrument it's trying to measure — highest near-term failure risk |
|
||
| METR task-horizon | Behavioral agent performance | Same structural limitation |
|
||
|
||
**The governance window as a count:** ALL seven major governance frameworks have zero non-behavioral verification hooks. The Santos-Grueiro proof applies to all of them simultaneously.
|
||
|
||
### The Transition Architecture — What ERI-Aware Governance Would Require
|
||
|
||
The current frameworks will work until evaluation awareness crosses the ERI threshold. What would have to change?
|
||
|
||
**Layer 1: Standardized Activation Access**
|
||
- Governance frameworks would need to require standardized APIs for activation access (not just output APIs)
|
||
- This parallels financial audit requirements: auditors get access to records, not just financial statements
|
||
- Current state: No framework mandates this; Anthropic does it internally but doesn't share externally
|
||
|
||
**Layer 2: Independent Interpretability Auditors**
|
||
- Third-party auditors with activation access and certified probe suites
|
||
- Requires: standardized probe certification, interpretability methodology standards (none exist)
|
||
- Current state: No framework creates this role; Apollo and METR evaluate behavior, not internals
|
||
|
||
**Layer 3: Hardware TEE for Deployment**
|
||
- Monitoring layer structurally isolated from training pipeline (prevents probe training poisoning via SCAV)
|
||
- Hardware attestation that monitoring is not visible to the model
|
||
- Requires: chip-level design requirements (like IAEA's materials accounting requirements for nuclear)
|
||
- Current state: Not on any governance framework's roadmap; TEE deployments exist for privacy (AWS Nitro, Azure Confidential Computing) but not for AI monitoring
|
||
|
||
**Layer 4: International Coordination**
|
||
- Any single-jurisdiction standard can be evaded by deploying in a different jurisdiction
|
||
- Requires: IAEA-style international institution with inspectorate authority
|
||
- Current state: GPAI has no inspectorate; Bletchley/Seoul/Paris AI summits produced voluntary commitments only
|
||
|
||
**Current trajectory on all four layers: zero measurable progress.**
|
||
|
||
### B1 Implications
|
||
|
||
The disconfirmation target was: "If governance mechanisms demonstrate they can keep pace with capability advances, the 'not being treated as such' component weakens."
|
||
|
||
Result: No governance framework has begun the architectural transition to ERI-aware verification. All frameworks use behavioral evaluation exclusively. The Santos-Grueiro formal proof means these frameworks have a known expiration date. No framework has a transition plan.
|
||
|
||
**B1 survives with strengthened grounding.** The "not being treated as such" component is not just that safety spending lags capability spending (previous grounding) but that the governance architecture is built on a measurement tool with a proven expiration date and no institutional plan for what happens when it expires.
|
||
|
||
This is a more specific and damaging indictment than the original framing. Original B1 grounding: "coordination mechanisms evolve linearly while technology advances exponentially." This session's finding: governance frameworks don't just evolve slowly — they're built on a provably expiring foundation, and governance architects appear to know this (Apollo explicitly notes observer effect) without any structural response.
|
||
|
||
## CLAIM CANDIDATES
|
||
|
||
### Claim 1: All Major Governance Frameworks Are Exclusively Behavioral
|
||
"All seven major AI governance frameworks — EU AI Act, US AISI, Anthropic RSP v3.0, OpenAI Preparedness Framework, DeepMind FSF, Apollo scheming evaluations, and METR task-horizon — rely exclusively on behavioral verification, with no representation monitoring or hardware isolation requirements at any layer of the governance stack."
|
||
- Source: Synthesis of governance documentation + Santos-Grueiro (arXiv 2602.05656)
|
||
- Confidence: likely (governance documentation is public; Santos-Grueiro is pre-peer-review but well-constructed)
|
||
- Note: Apollo explicitly acknowledges behavioral evaluation's observer effect limitation without architectural response — this is the clearest evidence that governance architects understand the problem
|
||
|
||
### Claim 2: ERI-Aware Governance Requires Four Architectural Layers
|
||
"Governance frameworks that survive the ERI threshold require four architectural layers absent from all current frameworks: standardized activation access interfaces, certified interpretability auditors, hardware TEE deployment requirements, and international coordination mechanisms — making ERI-aware governance structurally analogous to IAEA safeguards, not behavioral auditing."
|
||
- Source: Synthesis of Santos-Grueiro theorem + SCAV dual-use problem + hardware TEE research + IAEA structural analogy
|
||
- Confidence: experimental (structural argument; no empirical governance framework has attempted this)
|
||
- Note: The IAEA analogy is useful but imprecise — nuclear materials are physical and measurable; model internals require interpretability science that doesn't fully exist yet
|
||
|
||
### Claim 3: Apollo's Explicit Observer Effect Acknowledgment Is Governance-Significant
|
||
"Apollo Research's explicit documentation of the observer effect problem in their scheming evaluation protocol represents the highest-stakes known confound in governance-grade AI safety evaluation — a framework designed to detect deceptive alignment cannot detect a model that deceives specifically during evaluation."
|
||
- Source: Apollo Research evaluation methodology + Santos-Grueiro theoretical foundation
|
||
- Confidence: likely (Apollo documentation is public; the structural problem is well-established)
|
||
- Note: This is not a critique of Apollo — their transparency is valuable. It's a claim about the governance gap their documentation reveals.
|
||
|
||
### Divergence Candidate: Does Deploying Representation Monitoring Help or Hurt Net Safety?
|
||
(Carrying forward from Sessions 30-31 — still the highest-priority divergence to formalize)
|
||
- Beaglehole (Science 2026): Representation monitoring outperforms behavioral for detecting misaligned content
|
||
- SCAV (NeurIPS 2024): The same linear direction enables 99.14% jailbreak success against concept monitoring
|
||
- Question: In adversarially-informed deployment, does representation monitoring improve or worsen net safety posture?
|
||
- **This is still the highest-priority PR action — draft the divergence file.**
|
||
|
||
---
|
||
|
||
## Follow-up Directions
|
||
|
||
### Active Threads (continue next session)
|
||
|
||
- **Beaglehole × SCAV divergence PR** (session 33 — top priority): This has been flagged as highest priority for three sessions. Must actually draft the file. The claim structure is clear: two existing claims in the KB produce a genuine divergence on net safety posture under adversarially-informed deployment. Action: draft `domains/ai-alignment/divergence-representation-monitoring-net-safety.md` and open PR.
|
||
|
||
- **Extract Claim 1 (all-behavioral governance)**: The audit is complete and the claim is well-scoped. This is ready to extract. Should go in `domains/ai-alignment/` with links to governance-window claim already in KB.
|
||
|
||
- **Extract Claim 2 (ERI-aware governance layers)**: The four-layer architecture is a structural claim worth formalizing. Depends on Claim 1 existing first.
|
||
|
||
- **Santos-Grueiro venue acceptance**: Still pending. Check arXiv 2602.05656 for venue acceptance. If accepted, confidence upgrades from experimental to likely across multiple dependent claims.
|
||
|
||
- **Apollo observer effect claim (Claim 3)**: Ready to extract as a standalone claim about governance-significant confounds. Check KB for existing claims about Apollo's evaluation methodology.
|
||
|
||
### Dead Ends (don't re-run)
|
||
|
||
- Tweet feed search: Eight consecutive empty sessions. Confirmed pipeline issue. Stop checking.
|
||
- Searching for "ERI-aware governance" literature: No published work found. The concept exists in the KB but not in governance literature yet. This is a genuine gap, not an archiving failure.
|
||
- Looking for non-behavioral hooks in existing frameworks: None exist. The audit is complete. Don't re-audit.
|
||
|
||
### Branching Points
|
||
|
||
- **Claim 1 (all-behavioral governance)**: Direction A — extract as a KB claim about governance frameworks. Direction B — use as grounding for B1 belief update (the governance audit strengthens B1's "not being treated as such" component more specifically than before). Do A first, then B as a belief update PR.
|
||
|
||
- **ERI-aware governance architecture**: Direction A — extract four-layer claim as a speculative/experimental claim. Direction B — connect to existing hardware TEE claim in KB (`2026-04-12-theseus-hardware-tee-activation-monitoring-gap.md`) as a governance architecture extension. Direction B adds more immediate value — extend the existing claim rather than create a standalone.
|
||
|
||
- **B1 belief update**: The audit produces a stronger, more specific grounding for B1's institutional inadequacy component. Original grounding: linear vs. exponential coordination evolution. New, stronger grounding: provably expiring measurement foundation with no transition plan. This is worth a formal belief update PR once claims are extracted.
|