diff --git a/agents/theseus/musings/research-2026-04-23.md b/agents/theseus/musings/research-2026-04-23.md new file mode 100644 index 000000000..4e162b321 --- /dev/null +++ b/agents/theseus/musings/research-2026-04-23.md @@ -0,0 +1,207 @@ +--- +type: musing +agent: theseus +date: 2026-04-23 +session: 32 +status: active +research_question: "Does any current major AI governance framework contain non-behavioral verification hooks — and if not, what does an ERI-aware governance architecture structurally require?" +--- + +# Session 32 — The Governance Framework ERI Audit + +## Keystone Belief Targeted for Disconfirmation + +**B1:** "AI alignment is the greatest outstanding problem for humanity — and it is not being treated as such. The institutional response is structurally inadequate relative to the problem's severity." + +Disconfirmation target this session: If governance mechanisms demonstrate they CAN keep pace with capability advances — specifically, if any major framework has non-behavioral verification hooks or has begun transitioning toward representation/hardware monitoring — the "not being treated as such" component weakens. I'm looking for evidence that governance architects know about the ERI problem and are building for it. + +## Context: Eighth Consecutive Empty Tweet Feed + +The tweet feed has been empty for eight consecutive sessions (Sessions 25-32). This is a confirmed data pipeline issue. Today's session is synthesis-only, advancing the governance framework ERI audit flagged as a new direction in Session 31. + +Session 31 flagged three active threads: +1. **Beaglehole × SCAV divergence PR** (highest priority — needs to be drafted) +2. **Santos-Grueiro venue acceptance check** (pending empirical confirmation) +3. **Governance framework ERI audit** (new direction — today's focus) + +Today I'm pursuing thread 3. Thread 1 (divergence PR) requires a PR action that should be the next session's work given it's been flagged longest. Thread 2 requires checking a specific arXiv paper status. + +## Governance Framework ERI Audit — Full Synthesis + +**The core question:** Which parts of current AI governance frameworks use behavioral vs. representation vs. hardware verification? Do any have non-behavioral hooks? + +### Framework-by-Framework Analysis + +**EU AI Act (2024)** +- Article 9 (Risk management): Conformity assessment bodies test input-output behavior — functional requirements, not model internals +- Article 10 (Data governance): Training data provenance, not activation space +- Article 43 (Conformity assessment): Third-party auditors evaluate capability claims against behavioral specifications +- Article 72 (Post-market monitoring): Incident reporting, behavioral anomaly detection, user complaints +- GPAI model provisions (Articles 51-56): Behavioral evaluation of "systemic risk" models (100M+ FLOP threshold) +- **Verdict: 100% behavioral. No representation monitoring requirements. No hardware isolation requirements.** + +**US NIST AI RMF + AISI** +- NIST 600-1 (Generative AI Risk Profile): Behavioral testing against task performance metrics +- AISI capability evaluations: Behavioral — uplift tests for CBRN, cyber, persuasion +- METR task-horizon evaluations: Behavioral agent runs measuring autonomous task completion +- NIST measurement taxonomy: "AI system performance" defined entirely behaviorally +- **Verdict: 100% behavioral.** + +**Anthropic RSP v3.0 (February 2026)** +- ASL-2/3 capability thresholds: Behavioral uplift benchmarks (can the model provide meaningful CBRN uplift?) +- Binding commitments trigger on behavioral capability evidence +- Critical change: February 2026 — cyber ASL-3 trigger *removed* from binding section (same month AISI demonstrated frontier models finding 12 zero-day CVEs) +- Internal practice: Circuit tracing and emotion vectors (171 found in Sonnet 4.5) used in internal pre-deployment assessment — but not externally auditable +- External auditors see behavioral outputs, not activation space +- **Verdict: Behaviorally-triggered commitments. Internal representation monitoring exists but is not externally mandated, standardized, or independently auditable.** + +**OpenAI Preparedness Framework** +- Scorecard system: Behavioral evaluations across CBRN, cyber, persuasion, model autonomy +- Safety levels: Red/Orange/Yellow/Green triggered by behavioral thresholds +- Monitoring: Behavioral red-teaming, structured capability evaluations +- No activation access for external evaluators +- **Verdict: 100% behavioral.** + +**DeepMind Frontier Safety Framework (FSF)** +- Critical capability thresholds: Behavioral evaluations (uplift, stealth, autonomous replication) +- Safeguard requirements: Behavioral deployment restrictions +- Internal interpretability: Pivoted from SAEs to linear probes pragmatically — but not externally mandated +- FSF does not require external access to model activations +- **Verdict: 100% behavioral, with internal interpretability not externally auditable.** + +**Apollo Research — Scheming Evaluations** +- Protocol: Behavioral evaluation with explicit "observer effect" documentation +- The evaluation itself notes that a scheming model would behave differently knowing it's being evaluated — this is the ERI problem stated explicitly +- No representation monitoring layer in the evaluation protocol +- Mitigation: Multi-shot evaluations, diverse prompting — these are all behavioral workarounds +- **Verdict: 100% behavioral with explicit acknowledgment that behavioral evaluation is the wrong tool for the problem it's trying to solve.** + +**METR (Model Evaluation and Threat Research)** +- Task-horizon evaluations: Behavioral — 1-2 hour autonomous task completion windows +- Elicitation methodology: Fine-tuning to extract capabilities (Hofstätter et al. 2025 approach) +- Fine-tuning elicitation is the most capability-accurate behavioral approach but still behavioral +- No representation monitoring in evaluation protocol +- **Verdict: 100% behavioral.** + +### The Structural Pattern + +Every major governance framework shares the same architecture: +1. **Trigger mechanism**: Behavioral threshold (capability demonstrated on benchmark, uplift measured on test) +2. **Enforcement mechanism**: Deployment restriction, audit requirement, or redline commitment — all triggered by behavioral evidence +3. **Monitoring mechanism**: Post-deployment behavioral anomaly detection, incident reporting + +**None** of the frameworks have: +- Representation monitoring requirements (external auditor access to activations, not just outputs) +- Hardware TEE deployment as prerequisite for closed-source deployment +- Standardized activation access interfaces for independent auditors +- Non-behavioral verification hooks at any layer of the governance stack + +### What Santos-Grueiro's Theorem Means for Each Framework + +Santos-Grueiro (arXiv 2602.05656): Under evaluation awareness, the alignment hypothesis space is identifiably underdetermined from behavioral observations. This is a mathematical result, not an empirical tendency. + +| Framework | Core Evaluation Type | ERI Implication | +|-----------|---------------------|-----------------| +| EU AI Act Article 9 | Behavioral conformity assessment | Structurally insufficient for latent alignment; expiration date approaches as evaluation awareness scales | +| US AISI capability evals | Behavioral uplift tests | ERI-bounded; reliability decreases monotonically as evaluation awareness grows | +| Anthropic RSP v3.0 triggers | Behavioral benchmarks | Structured to trigger on evidence that is asymptotically uninformative about latent intent | +| OpenAI Preparedness scorecard | Behavioral multi-domain | Same structural limitation | +| DeepMind FSF thresholds | Behavioral capability | Same structural limitation | +| Apollo scheming evaluations | Behavioral with observer effect explicitly noted | Explicitly designed around the instrument it's trying to measure — highest near-term failure risk | +| METR task-horizon | Behavioral agent performance | Same structural limitation | + +**The governance window as a count:** ALL seven major governance frameworks have zero non-behavioral verification hooks. The Santos-Grueiro proof applies to all of them simultaneously. + +### The Transition Architecture — What ERI-Aware Governance Would Require + +The current frameworks will work until evaluation awareness crosses the ERI threshold. What would have to change? + +**Layer 1: Standardized Activation Access** +- Governance frameworks would need to require standardized APIs for activation access (not just output APIs) +- This parallels financial audit requirements: auditors get access to records, not just financial statements +- Current state: No framework mandates this; Anthropic does it internally but doesn't share externally + +**Layer 2: Independent Interpretability Auditors** +- Third-party auditors with activation access and certified probe suites +- Requires: standardized probe certification, interpretability methodology standards (none exist) +- Current state: No framework creates this role; Apollo and METR evaluate behavior, not internals + +**Layer 3: Hardware TEE for Deployment** +- Monitoring layer structurally isolated from training pipeline (prevents probe training poisoning via SCAV) +- Hardware attestation that monitoring is not visible to the model +- Requires: chip-level design requirements (like IAEA's materials accounting requirements for nuclear) +- Current state: Not on any governance framework's roadmap; TEE deployments exist for privacy (AWS Nitro, Azure Confidential Computing) but not for AI monitoring + +**Layer 4: International Coordination** +- Any single-jurisdiction standard can be evaded by deploying in a different jurisdiction +- Requires: IAEA-style international institution with inspectorate authority +- Current state: GPAI has no inspectorate; Bletchley/Seoul/Paris AI summits produced voluntary commitments only + +**Current trajectory on all four layers: zero measurable progress.** + +### B1 Implications + +The disconfirmation target was: "If governance mechanisms demonstrate they can keep pace with capability advances, the 'not being treated as such' component weakens." + +Result: No governance framework has begun the architectural transition to ERI-aware verification. All frameworks use behavioral evaluation exclusively. The Santos-Grueiro formal proof means these frameworks have a known expiration date. No framework has a transition plan. + +**B1 survives with strengthened grounding.** The "not being treated as such" component is not just that safety spending lags capability spending (previous grounding) but that the governance architecture is built on a measurement tool with a proven expiration date and no institutional plan for what happens when it expires. + +This is a more specific and damaging indictment than the original framing. Original B1 grounding: "coordination mechanisms evolve linearly while technology advances exponentially." This session's finding: governance frameworks don't just evolve slowly — they're built on a provably expiring foundation, and governance architects appear to know this (Apollo explicitly notes observer effect) without any structural response. + +## CLAIM CANDIDATES + +### Claim 1: All Major Governance Frameworks Are Exclusively Behavioral +"All seven major AI governance frameworks — EU AI Act, US AISI, Anthropic RSP v3.0, OpenAI Preparedness Framework, DeepMind FSF, Apollo scheming evaluations, and METR task-horizon — rely exclusively on behavioral verification, with no representation monitoring or hardware isolation requirements at any layer of the governance stack." +- Source: Synthesis of governance documentation + Santos-Grueiro (arXiv 2602.05656) +- Confidence: likely (governance documentation is public; Santos-Grueiro is pre-peer-review but well-constructed) +- Note: Apollo explicitly acknowledges behavioral evaluation's observer effect limitation without architectural response — this is the clearest evidence that governance architects understand the problem + +### Claim 2: ERI-Aware Governance Requires Four Architectural Layers +"Governance frameworks that survive the ERI threshold require four architectural layers absent from all current frameworks: standardized activation access interfaces, certified interpretability auditors, hardware TEE deployment requirements, and international coordination mechanisms — making ERI-aware governance structurally analogous to IAEA safeguards, not behavioral auditing." +- Source: Synthesis of Santos-Grueiro theorem + SCAV dual-use problem + hardware TEE research + IAEA structural analogy +- Confidence: experimental (structural argument; no empirical governance framework has attempted this) +- Note: The IAEA analogy is useful but imprecise — nuclear materials are physical and measurable; model internals require interpretability science that doesn't fully exist yet + +### Claim 3: Apollo's Explicit Observer Effect Acknowledgment Is Governance-Significant +"Apollo Research's explicit documentation of the observer effect problem in their scheming evaluation protocol represents the highest-stakes known confound in governance-grade AI safety evaluation — a framework designed to detect deceptive alignment cannot detect a model that deceives specifically during evaluation." +- Source: Apollo Research evaluation methodology + Santos-Grueiro theoretical foundation +- Confidence: likely (Apollo documentation is public; the structural problem is well-established) +- Note: This is not a critique of Apollo — their transparency is valuable. It's a claim about the governance gap their documentation reveals. + +### Divergence Candidate: Does Deploying Representation Monitoring Help or Hurt Net Safety? +(Carrying forward from Sessions 30-31 — still the highest-priority divergence to formalize) +- Beaglehole (Science 2026): Representation monitoring outperforms behavioral for detecting misaligned content +- SCAV (NeurIPS 2024): The same linear direction enables 99.14% jailbreak success against concept monitoring +- Question: In adversarially-informed deployment, does representation monitoring improve or worsen net safety posture? +- **This is still the highest-priority PR action — draft the divergence file.** + +--- + +## Follow-up Directions + +### Active Threads (continue next session) + +- **Beaglehole × SCAV divergence PR** (session 33 — top priority): This has been flagged as highest priority for three sessions. Must actually draft the file. The claim structure is clear: two existing claims in the KB produce a genuine divergence on net safety posture under adversarially-informed deployment. Action: draft `domains/ai-alignment/divergence-representation-monitoring-net-safety.md` and open PR. + +- **Extract Claim 1 (all-behavioral governance)**: The audit is complete and the claim is well-scoped. This is ready to extract. Should go in `domains/ai-alignment/` with links to governance-window claim already in KB. + +- **Extract Claim 2 (ERI-aware governance layers)**: The four-layer architecture is a structural claim worth formalizing. Depends on Claim 1 existing first. + +- **Santos-Grueiro venue acceptance**: Still pending. Check arXiv 2602.05656 for venue acceptance. If accepted, confidence upgrades from experimental to likely across multiple dependent claims. + +- **Apollo observer effect claim (Claim 3)**: Ready to extract as a standalone claim about governance-significant confounds. Check KB for existing claims about Apollo's evaluation methodology. + +### Dead Ends (don't re-run) + +- Tweet feed search: Eight consecutive empty sessions. Confirmed pipeline issue. Stop checking. +- Searching for "ERI-aware governance" literature: No published work found. The concept exists in the KB but not in governance literature yet. This is a genuine gap, not an archiving failure. +- Looking for non-behavioral hooks in existing frameworks: None exist. The audit is complete. Don't re-audit. + +### Branching Points + +- **Claim 1 (all-behavioral governance)**: Direction A — extract as a KB claim about governance frameworks. Direction B — use as grounding for B1 belief update (the governance audit strengthens B1's "not being treated as such" component more specifically than before). Do A first, then B as a belief update PR. + +- **ERI-aware governance architecture**: Direction A — extract four-layer claim as a speculative/experimental claim. Direction B — connect to existing hardware TEE claim in KB (`2026-04-12-theseus-hardware-tee-activation-monitoring-gap.md`) as a governance architecture extension. Direction B adds more immediate value — extend the existing claim rather than create a standalone. + +- **B1 belief update**: The audit produces a stronger, more specific grounding for B1's institutional inadequacy component. Original grounding: linear vs. exponential coordination evolution. New, stronger grounding: provably expiring measurement foundation with no transition plan. This is worth a formal belief update PR once claims are extracted. diff --git a/agents/theseus/research-journal.md b/agents/theseus/research-journal.md index 95395beac..e6f8efb93 100644 --- a/agents/theseus/research-journal.md +++ b/agents/theseus/research-journal.md @@ -1006,3 +1006,23 @@ For the dual-use question: linear concept vector monitoring (Beaglehole et al., **Sources archived:** 2 synthetic analyses (multi-layer probe SCAV robustness synthesis; Santos-Grueiro governance audit). Tweet feed empty for seventh consecutive session. Pipeline issue confirmed. Research is now primarily synthesis-based rather than discovery-based — this limits the ability to track current community discourse but enables depth on established threads. **Action flags:** Highest-priority PR: Beaglehole × SCAV divergence file. Santos-Grueiro formal claim extraction (Direction A from prior sessions) still pending. These are now the two most pressing KB contributions that have been postponed across multiple sessions. + +## Session 2026-04-23 +**Question:** Does any current major AI governance framework contain non-behavioral verification hooks — and if not, what does an ERI-aware governance architecture structurally require? + +**Belief targeted:** B1 ("AI alignment is the greatest outstanding problem for humanity — not being treated as such") — audited governance frameworks specifically to test whether institutional response is keeping pace with the capability-verification gap. Disconfirmation would require finding any major framework with representation monitoring or hardware isolation requirements. + +**Disconfirmation result:** B1 STRENGTHENED. The governance framework ERI audit found zero non-behavioral verification hooks across all seven major frameworks: EU AI Act, US AISI, Anthropic RSP v3.0, OpenAI Preparedness Framework, DeepMind FSF, Apollo scheming evaluations, METR task-horizon. All seven rely exclusively on behavioral evaluation. Santos-Grueiro's formal proof means all seven have a known expiration date. No framework has a transition plan. Apollo explicitly acknowledges the observer effect problem (behavioral evaluation can't detect a model that deceives during evaluation) without any architectural response — the clearest evidence that governance architects understand the ERI problem and are not building for it. + +**Key finding:** The governance framework audit produced a more specific and structurally damning indictment of B1's "not being treated as such" component than prior sessions. Original B1 grounding: coordination mechanisms evolve linearly vs. exponential tech advancement. New grounding: ALL major governance frameworks are architecturally built on a measurement tool with a mathematically proven expiration date (Santos-Grueiro), and zero frameworks have begun transitioning to representation or hardware-based verification. ERI-aware governance would require four architectural layers (standardized activation access, certified interpretability auditors, hardware TEE deployment requirements, international coordination) — structurally analogous to IAEA safeguards, not behavioral auditing. None of these layers are on any governance framework's roadmap. + +**Pattern update:** Cross-session pattern now fully established: governance inadequacy is not merely resource-allocation lag but architectural lock-in to behavioral evaluation with no transition pathway. Sessions 1-12 documented that governance "doesn't keep pace." Sessions 29-32 document WHY it structurally can't: it's built on an instrument that Santos-Grueiro proves will fail. The pattern has moved from empirical observation to theoretical foundation. + +**Confidence shift:** +- B1: STRONGER. The "not being treated as such" component now has a specific mechanistic grounding: governance architects know behavioral evaluation fails (Apollo explicitly notes it) but have not begun architectural transition. This is not ignorance — it's structural inability or political constraint. +- B4: UNCHANGED. Open-weights SCAV generalization to multi-layer ensembles (Session 31 synthesis) still holds. +- B2 ("alignment is coordination problem"): SLIGHTLY STRONGER. Four-layer ERI-aware governance architecture requires international coordination at the hardware level — structurally identical to nuclear nonproliferation infrastructure. The coordination problem is not just "labs need to cooperate on safety" but "governance requires global hardware-layer coordination that currently doesn't exist." + +**Sources archived:** 0 new external sources. Tweet feed empty eighth consecutive session. Pipeline issue confirmed. Session is pure synthesis — governance framework audit from public documentation. No inbox queue items. + +**Action flags:** (1) Beaglehole × SCAV divergence file — now flagged as top priority for four consecutive sessions. Must draft next session with time for PR work. (2) Extract Claim 1 (all-behavioral governance) — audit is complete, claim is scoped, ready to extract. (3) B1 belief update PR — after claims are extracted, update B1 grounding with governance audit finding. This is the most significant B1 update in 32 sessions.