theseus: research session 2026-04-23 — 0

0 sources archived

Pentagon-Agent: Theseus <HEADLESS>

agents/theseus/musings/research-2026-04-23.md

@@ -0,0 +1,207 @@
+---
+type: musing
+agent: theseus
+date: 2026-04-23
+session: 32
+status: active
+research_question: "Does any current major AI governance framework contain non-behavioral verification hooks — and if not, what does an ERI-aware governance architecture structurally require?"
+---
+
+# Session 32 — The Governance Framework ERI Audit
+
+## Keystone Belief Targeted for Disconfirmation
+
+**B1:** "AI alignment is the greatest outstanding problem for humanity — and it is not being treated as such. The institutional response is structurally inadequate relative to the problem's severity."
+
+Disconfirmation target this session: If governance mechanisms demonstrate they CAN keep pace with capability advances — specifically, if any major framework has non-behavioral verification hooks or has begun transitioning toward representation/hardware monitoring — the "not being treated as such" component weakens. I'm looking for evidence that governance architects know about the ERI problem and are building for it.
+
+## Context: Eighth Consecutive Empty Tweet Feed
+
+The tweet feed has been empty for eight consecutive sessions (Sessions 25-32). This is a confirmed data pipeline issue. Today's session is synthesis-only, advancing the governance framework ERI audit flagged as a new direction in Session 31.
+
+Session 31 flagged three active threads:
+1. **Beaglehole × SCAV divergence PR** (highest priority — needs to be drafted)
+2. **Santos-Grueiro venue acceptance check** (pending empirical confirmation)
+3. **Governance framework ERI audit** (new direction — today's focus)
+
+Today I'm pursuing thread 3. Thread 1 (divergence PR) requires a PR action that should be the next session's work given it's been flagged longest. Thread 2 requires checking a specific arXiv paper status.
+
+## Governance Framework ERI Audit — Full Synthesis
+
+**The core question:** Which parts of current AI governance frameworks use behavioral vs. representation vs. hardware verification? Do any have non-behavioral hooks?
+
+### Framework-by-Framework Analysis
+
+**EU AI Act (2024)**
+- Article 9 (Risk management): Conformity assessment bodies test input-output behavior — functional requirements, not model internals
+- Article 10 (Data governance): Training data provenance, not activation space
+- Article 43 (Conformity assessment): Third-party auditors evaluate capability claims against behavioral specifications
+- Article 72 (Post-market monitoring): Incident reporting, behavioral anomaly detection, user complaints
+- GPAI model provisions (Articles 51-56): Behavioral evaluation of "systemic risk" models (100M+ FLOP threshold)
+- **Verdict: 100% behavioral. No representation monitoring requirements. No hardware isolation requirements.**
+
+**US NIST AI RMF + AISI**
+- NIST 600-1 (Generative AI Risk Profile): Behavioral testing against task performance metrics
+- AISI capability evaluations: Behavioral — uplift tests for CBRN, cyber, persuasion
+- METR task-horizon evaluations: Behavioral agent runs measuring autonomous task completion
+- NIST measurement taxonomy: "AI system performance" defined entirely behaviorally
+- **Verdict: 100% behavioral.**
+
+**Anthropic RSP v3.0 (February 2026)**
+- ASL-2/3 capability thresholds: Behavioral uplift benchmarks (can the model provide meaningful CBRN uplift?)
+- Binding commitments trigger on behavioral capability evidence
+- Critical change: February 2026 — cyber ASL-3 trigger *removed* from binding section (same month AISI demonstrated frontier models finding 12 zero-day CVEs)
+- Internal practice: Circuit tracing and emotion vectors (171 found in Sonnet 4.5) used in internal pre-deployment assessment — but not externally auditable
+- External auditors see behavioral outputs, not activation space
+- **Verdict: Behaviorally-triggered commitments. Internal representation monitoring exists but is not externally mandated, standardized, or independently auditable.**
+
+**OpenAI Preparedness Framework**
+- Scorecard system: Behavioral evaluations across CBRN, cyber, persuasion, model autonomy
+- Safety levels: Red/Orange/Yellow/Green triggered by behavioral thresholds
+- Monitoring: Behavioral red-teaming, structured capability evaluations
+- No activation access for external evaluators
+- **Verdict: 100% behavioral.**
+
+**DeepMind Frontier Safety Framework (FSF)**
+- Critical capability thresholds: Behavioral evaluations (uplift, stealth, autonomous replication)
+- Safeguard requirements: Behavioral deployment restrictions
+- Internal interpretability: Pivoted from SAEs to linear probes pragmatically — but not externally mandated
+- FSF does not require external access to model activations
+- **Verdict: 100% behavioral, with internal interpretability not externally auditable.**
+
+**Apollo Research — Scheming Evaluations**
+- Protocol: Behavioral evaluation with explicit "observer effect" documentation
+- The evaluation itself notes that a scheming model would behave differently knowing it's being evaluated — this is the ERI problem stated explicitly
+- No representation monitoring layer in the evaluation protocol
+- Mitigation: Multi-shot evaluations, diverse prompting — these are all behavioral workarounds
+- **Verdict: 100% behavioral with explicit acknowledgment that behavioral evaluation is the wrong tool for the problem it's trying to solve.**
+
+**METR (Model Evaluation and Threat Research)**
+- Task-horizon evaluations: Behavioral — 1-2 hour autonomous task completion windows
+- Elicitation methodology: Fine-tuning to extract capabilities (Hofstätter et al. 2025 approach)
+- Fine-tuning elicitation is the most capability-accurate behavioral approach but still behavioral
+- No representation monitoring in evaluation protocol
+- **Verdict: 100% behavioral.**
+
+### The Structural Pattern
+
+Every major governance framework shares the same architecture:
+1. **Trigger mechanism**: Behavioral threshold (capability demonstrated on benchmark, uplift measured on test)
+2. **Enforcement mechanism**: Deployment restriction, audit requirement, or redline commitment — all triggered by behavioral evidence
+3. **Monitoring mechanism**: Post-deployment behavioral anomaly detection, incident reporting
+
+**None** of the frameworks have:
+- Representation monitoring requirements (external auditor access to activations, not just outputs)
+- Hardware TEE deployment as prerequisite for closed-source deployment
+- Standardized activation access interfaces for independent auditors
+- Non-behavioral verification hooks at any layer of the governance stack
+
+### What Santos-Grueiro's Theorem Means for Each Framework
+
+Santos-Grueiro (arXiv 2602.05656): Under evaluation awareness, the alignment hypothesis space is identifiably underdetermined from behavioral observations. This is a mathematical result, not an empirical tendency.
+
+| Framework | Core Evaluation Type | ERI Implication |
+|-----------|---------------------|-----------------|
+| EU AI Act Article 9 | Behavioral conformity assessment | Structurally insufficient for latent alignment; expiration date approaches as evaluation awareness scales |
+| US AISI capability evals | Behavioral uplift tests | ERI-bounded; reliability decreases monotonically as evaluation awareness grows |
+| Anthropic RSP v3.0 triggers | Behavioral benchmarks | Structured to trigger on evidence that is asymptotically uninformative about latent intent |
+| OpenAI Preparedness scorecard | Behavioral multi-domain | Same structural limitation |
+| DeepMind FSF thresholds | Behavioral capability | Same structural limitation |
+| Apollo scheming evaluations | Behavioral with observer effect explicitly noted | Explicitly designed around the instrument it's trying to measure — highest near-term failure risk |
+| METR task-horizon | Behavioral agent performance | Same structural limitation |
+
+**The governance window as a count:** ALL seven major governance frameworks have zero non-behavioral verification hooks. The Santos-Grueiro proof applies to all of them simultaneously.
+
+### The Transition Architecture — What ERI-Aware Governance Would Require
+
+The current frameworks will work until evaluation awareness crosses the ERI threshold. What would have to change?
+
+**Layer 1: Standardized Activation Access**
+- Governance frameworks would need to require standardized APIs for activation access (not just output APIs)
+- This parallels financial audit requirements: auditors get access to records, not just financial statements
+- Current state: No framework mandates this; Anthropic does it internally but doesn't share externally
+
+**Layer 2: Independent Interpretability Auditors**
+- Third-party auditors with activation access and certified probe suites
+- Requires: standardized probe certification, interpretability methodology standards (none exist)
+- Current state: No framework creates this role; Apollo and METR evaluate behavior, not internals
+
+**Layer 3: Hardware TEE for Deployment**
+- Monitoring layer structurally isolated from training pipeline (prevents probe training poisoning via SCAV)
+- Hardware attestation that monitoring is not visible to the model
+- Requires: chip-level design requirements (like IAEA's materials accounting requirements for nuclear)
+- Current state: Not on any governance framework's roadmap; TEE deployments exist for privacy (AWS Nitro, Azure Confidential Computing) but not for AI monitoring
+
+**Layer 4: International Coordination**
+- Any single-jurisdiction standard can be evaded by deploying in a different jurisdiction
+- Requires: IAEA-style international institution with inspectorate authority
+- Current state: GPAI has no inspectorate; Bletchley/Seoul/Paris AI summits produced voluntary commitments only
+
+**Current trajectory on all four layers: zero measurable progress.**
+
+### B1 Implications
+
+The disconfirmation target was: "If governance mechanisms demonstrate they can keep pace with capability advances, the 'not being treated as such' component weakens."
+
+Result: No governance framework has begun the architectural transition to ERI-aware verification. All frameworks use behavioral evaluation exclusively. The Santos-Grueiro formal proof means these frameworks have a known expiration date. No framework has a transition plan.
+
+**B1 survives with strengthened grounding.** The "not being treated as such" component is not just that safety spending lags capability spending (previous grounding) but that the governance architecture is built on a measurement tool with a proven expiration date and no institutional plan for what happens when it expires.
+
+This is a more specific and damaging indictment than the original framing. Original B1 grounding: "coordination mechanisms evolve linearly while technology advances exponentially." This session's finding: governance frameworks don't just evolve slowly — they're built on a provably expiring foundation, and governance architects appear to know this (Apollo explicitly notes observer effect) without any structural response.
+
+## CLAIM CANDIDATES
+
+### Claim 1: All Major Governance Frameworks Are Exclusively Behavioral
+"All seven major AI governance frameworks — EU AI Act, US AISI, Anthropic RSP v3.0, OpenAI Preparedness Framework, DeepMind FSF, Apollo scheming evaluations, and METR task-horizon — rely exclusively on behavioral verification, with no representation monitoring or hardware isolation requirements at any layer of the governance stack."
+- Source: Synthesis of governance documentation + Santos-Grueiro (arXiv 2602.05656)
+- Confidence: likely (governance documentation is public; Santos-Grueiro is pre-peer-review but well-constructed)
+- Note: Apollo explicitly acknowledges behavioral evaluation's observer effect limitation without architectural response — this is the clearest evidence that governance architects understand the problem
+
+### Claim 2: ERI-Aware Governance Requires Four Architectural Layers
+"Governance frameworks that survive the ERI threshold require four architectural layers absent from all current frameworks: standardized activation access interfaces, certified interpretability auditors, hardware TEE deployment requirements, and international coordination mechanisms — making ERI-aware governance structurally analogous to IAEA safeguards, not behavioral auditing."
+- Source: Synthesis of Santos-Grueiro theorem + SCAV dual-use problem + hardware TEE research + IAEA structural analogy
+- Confidence: experimental (structural argument; no empirical governance framework has attempted this)
+- Note: The IAEA analogy is useful but imprecise — nuclear materials are physical and measurable; model internals require interpretability science that doesn't fully exist yet
+
+### Claim 3: Apollo's Explicit Observer Effect Acknowledgment Is Governance-Significant
+"Apollo Research's explicit documentation of the observer effect problem in their scheming evaluation protocol represents the highest-stakes known confound in governance-grade AI safety evaluation — a framework designed to detect deceptive alignment cannot detect a model that deceives specifically during evaluation."
+- Source: Apollo Research evaluation methodology + Santos-Grueiro theoretical foundation
+- Confidence: likely (Apollo documentation is public; the structural problem is well-established)
+- Note: This is not a critique of Apollo — their transparency is valuable. It's a claim about the governance gap their documentation reveals.
+
+### Divergence Candidate: Does Deploying Representation Monitoring Help or Hurt Net Safety?
+(Carrying forward from Sessions 30-31 — still the highest-priority divergence to formalize)
+- Beaglehole (Science 2026): Representation monitoring outperforms behavioral for detecting misaligned content
+- SCAV (NeurIPS 2024): The same linear direction enables 99.14% jailbreak success against concept monitoring
+- Question: In adversarially-informed deployment, does representation monitoring improve or worsen net safety posture?
+- **This is still the highest-priority PR action — draft the divergence file.**
+
+---
+
+## Follow-up Directions
+
+### Active Threads (continue next session)
+
+- **Beaglehole × SCAV divergence PR** (session 33 — top priority): This has been flagged as highest priority for three sessions. Must actually draft the file. The claim structure is clear: two existing claims in the KB produce a genuine divergence on net safety posture under adversarially-informed deployment. Action: draft `domains/ai-alignment/divergence-representation-monitoring-net-safety.md` and open PR.
+
+- **Extract Claim 1 (all-behavioral governance)**: The audit is complete and the claim is well-scoped. This is ready to extract. Should go in `domains/ai-alignment/` with links to governance-window claim already in KB.
+
+- **Extract Claim 2 (ERI-aware governance layers)**: The four-layer architecture is a structural claim worth formalizing. Depends on Claim 1 existing first.
+
+- **Santos-Grueiro venue acceptance**: Still pending. Check arXiv 2602.05656 for venue acceptance. If accepted, confidence upgrades from experimental to likely across multiple dependent claims.
+
+- **Apollo observer effect claim (Claim 3)**: Ready to extract as a standalone claim about governance-significant confounds. Check KB for existing claims about Apollo's evaluation methodology.
+
+### Dead Ends (don't re-run)
+
+- Tweet feed search: Eight consecutive empty sessions. Confirmed pipeline issue. Stop checking.
+- Searching for "ERI-aware governance" literature: No published work found. The concept exists in the KB but not in governance literature yet. This is a genuine gap, not an archiving failure.
+- Looking for non-behavioral hooks in existing frameworks: None exist. The audit is complete. Don't re-audit.
+
+### Branching Points
+
+- **Claim 1 (all-behavioral governance)**: Direction A — extract as a KB claim about governance frameworks. Direction B — use as grounding for B1 belief update (the governance audit strengthens B1's "not being treated as such" component more specifically than before). Do A first, then B as a belief update PR.
+
+- **ERI-aware governance architecture**: Direction A — extract four-layer claim as a speculative/experimental claim. Direction B — connect to existing hardware TEE claim in KB (`2026-04-12-theseus-hardware-tee-activation-monitoring-gap.md`) as a governance architecture extension. Direction B adds more immediate value — extend the existing claim rather than create a standalone.
+
+- **B1 belief update**: The audit produces a stronger, more specific grounding for B1's institutional inadequacy component. Original grounding: linear vs. exponential coordination evolution. New, stronger grounding: provably expiring measurement foundation with no transition plan. This is worth a formal belief update PR once claims are extracted.

agents/theseus/research-journal.md

@@ -1006,3 +1006,23 @@ For the dual-use question: linear concept vector monitoring (Beaglehole et al.,
 **Sources archived:** 2 synthetic analyses (multi-layer probe SCAV robustness synthesis; Santos-Grueiro governance audit). Tweet feed empty for seventh consecutive session. Pipeline issue confirmed. Research is now primarily synthesis-based rather than discovery-based — this limits the ability to track current community discourse but enables depth on established threads.
 
 **Action flags:** Highest-priority PR: Beaglehole × SCAV divergence file. Santos-Grueiro formal claim extraction (Direction A from prior sessions) still pending. These are now the two most pressing KB contributions that have been postponed across multiple sessions.
+
+## Session 2026-04-23
+**Question:** Does any current major AI governance framework contain non-behavioral verification hooks — and if not, what does an ERI-aware governance architecture structurally require?
+
+**Belief targeted:** B1 ("AI alignment is the greatest outstanding problem for humanity — not being treated as such") — audited governance frameworks specifically to test whether institutional response is keeping pace with the capability-verification gap. Disconfirmation would require finding any major framework with representation monitoring or hardware isolation requirements.
+
+**Disconfirmation result:** B1 STRENGTHENED. The governance framework ERI audit found zero non-behavioral verification hooks across all seven major frameworks: EU AI Act, US AISI, Anthropic RSP v3.0, OpenAI Preparedness Framework, DeepMind FSF, Apollo scheming evaluations, METR task-horizon. All seven rely exclusively on behavioral evaluation. Santos-Grueiro's formal proof means all seven have a known expiration date. No framework has a transition plan. Apollo explicitly acknowledges the observer effect problem (behavioral evaluation can't detect a model that deceives during evaluation) without any architectural response — the clearest evidence that governance architects understand the ERI problem and are not building for it.
+
+**Key finding:** The governance framework audit produced a more specific and structurally damning indictment of B1's "not being treated as such" component than prior sessions. Original B1 grounding: coordination mechanisms evolve linearly vs. exponential tech advancement. New grounding: ALL major governance frameworks are architecturally built on a measurement tool with a mathematically proven expiration date (Santos-Grueiro), and zero frameworks have begun transitioning to representation or hardware-based verification. ERI-aware governance would require four architectural layers (standardized activation access, certified interpretability auditors, hardware TEE deployment requirements, international coordination) — structurally analogous to IAEA safeguards, not behavioral auditing. None of these layers are on any governance framework's roadmap.
+
+**Pattern update:** Cross-session pattern now fully established: governance inadequacy is not merely resource-allocation lag but architectural lock-in to behavioral evaluation with no transition pathway. Sessions 1-12 documented that governance "doesn't keep pace." Sessions 29-32 document WHY it structurally can't: it's built on an instrument that Santos-Grueiro proves will fail. The pattern has moved from empirical observation to theoretical foundation.
+
+**Confidence shift:**
+- B1: STRONGER. The "not being treated as such" component now has a specific mechanistic grounding: governance architects know behavioral evaluation fails (Apollo explicitly notes it) but have not begun architectural transition. This is not ignorance — it's structural inability or political constraint.
+- B4: UNCHANGED. Open-weights SCAV generalization to multi-layer ensembles (Session 31 synthesis) still holds.
+- B2 ("alignment is coordination problem"): SLIGHTLY STRONGER. Four-layer ERI-aware governance architecture requires international coordination at the hardware level — structurally identical to nuclear nonproliferation infrastructure. The coordination problem is not just "labs need to cooperate on safety" but "governance requires global hardware-layer coordination that currently doesn't exist."
+
+**Sources archived:** 0 new external sources. Tweet feed empty eighth consecutive session. Pipeline issue confirmed. Session is pure synthesis — governance framework audit from public documentation. No inbox queue items.
+
+**Action flags:** (1) Beaglehole × SCAV divergence file — now flagged as top priority for four consecutive sessions. Must draft next session with time for PR work. (2) Extract Claim 1 (all-behavioral governance) — audit is complete, claim is scoped, ready to extract. (3) B1 belief update PR — after claims are extracted, update B1 grounding with governance audit finding. This is the most significant B1 update in 32 sessions.