theseus: extract claims from 2026-01-00-mechanistic-interpretability-2026-status-report.md

- Source: inbox/archive/2026-01-00-mechanistic-interpretability-2026-status-report.md
- Domain: ai-alignment
- Extracted by: headless extraction cron (worker 4)

Pentagon-Agent: Theseus <HEADLESS>

domains/ai-alignment/AI alignment is a coordination problem not a technical problem.md

@@ -21,6 +21,12 @@ Dario Amodei describes AI as "so powerful, such a glittering prize, that it is v
 
 Since [[the internet enabled global communication but not global cognition]], the coordination infrastructure needed doesn't exist yet. This is why [[collective superintelligence is the alternative to monolithic AI controlled by a few]] -- it solves alignment through architecture rather than attempting governance from outside the system.
 
+
+### Additional Evidence (confirm)
+*Source: [[2026-01-00-mechanistic-interpretability-2026-status-report]] | Added: 2026-03-11 | Extractor: anthropic/claude-sonnet-4.5*
+
+The 2026 interpretability status confirms that technical alignment (interpretability) is making genuine progress on diagnostic capabilities but cannot address coordination or preference diversity problems. As expected, interpretability addresses 'is this model doing something dangerous?' but provides no mechanism for 'is this model serving diverse values?' or 'are competing models producing safe interaction effects?' The field's convergence on diagnostic scope (Anthropic: 'reliably detecting most model problems by 2027') rather than comprehensive alignment validates the claim that technical approaches are bounded. Neel Nanda's assessment that 'the most ambitious vision...is probably dead' reflects the field's recognition that technical understanding alone cannot solve alignment.
+
 ---
 
 Relevant Notes:

domains/ai-alignment/SAE-reconstructions-degrade-model-performance-by-10-to-40-percent-making-interpretability-costly-at-deployment.md

@@ -0,0 +1,52 @@
+---
+type: claim
+domain: ai-alignment
+description: "Sparse autoencoder reconstructions cause substantial performance loss on downstream tasks creating a direct capability cost for interpretability"
+confidence: likely
+source: "2026 mechanistic interpretability status report, multiple lab findings"
+created: 2026-01-01
+last_evaluated: 2026-01-01
+depends_on:
+  - "the alignment tax creates a structural race to the bottom because safety training costs capability and rational competitors skip it"
+---
+
+# SAE reconstructions degrade model performance by 10 to 40 percent making interpretability costly at deployment
+
+Sparse autoencoders (SAEs), the primary tool for mechanistic interpretability, cause 10-40% performance degradation on downstream tasks when used to reconstruct model activations. This represents a direct capability cost that makes interpretability-in-the-loop deployment economically punishing in competitive markets.
+
+This performance degradation is distinct from the compute costs of interpretability — it's the cost paid even after interpretability infrastructure is built. A model using SAE-based interpretability for runtime safety checks would perform measurably worse than an identical model without interpretability, creating a structural disadvantage in competitive deployment.
+
+The degradation range (10-40%) varies by task and model architecture, but the consistency of the finding across multiple labs and model scales (from 270M to 27B parameters in Gemma Scope 2) suggests this is a fundamental limitation of current SAE approaches, not an implementation artifact.
+
+## Evidence
+
+**Performance degradation:**
+- SAE reconstructions cause 10-40% performance degradation on downstream tasks (2026 status report)
+- Finding consistent across Google DeepMind's Gemma Scope 2 (270M to 27B parameter models)
+- Degradation occurs even with SAEs scaled to GPT-4 with 16 million latent variables
+
+**Strategic implications:**
+- Google DeepMind found SAEs underperformed simple linear probes on practical safety tasks
+- This combination (performance cost + inferior safety detection) led to strategic pivot away from SAEs
+- Anthropic's production use suggests degradation is acceptable when interpretability provides deployment-critical insights
+
+**Mechanism:**
+- SAEs compress model activations into sparse representations
+- Reconstruction from sparse representations loses information
+- Information loss manifests as capability degradation on tasks requiring the lost information
+
+## Challenges
+
+The 10-40% range is wide, suggesting that degradation may be task-dependent or that some SAE architectures perform better than others. If degradation can be reduced to the lower end (10%) for safety-critical tasks, the cost may be acceptable for high-stakes deployment.
+
+Additionally, Anthropic's integration into Claude Sonnet 4.5 deployment suggests that the degradation is not prohibitive when interpretability provides unique value (legible explanations, mechanistic understanding) that baselines cannot match.
+
+---
+
+Relevant Notes:
+- [[the alignment tax creates a structural race to the bottom because safety training costs capability and rational competitors skip it]] — 10-40% degradation is a quantified alignment tax
+- [[interpretability compute costs amplify the alignment tax through massive resource requirements]] — performance degradation is in addition to compute costs
+- [[voluntary safety pledges cannot survive competitive pressure because unilateral commitments are structurally punished when competitors advance without equivalent constraints]] — applies to interpretability adoption
+
+Topics:
+- [[domains/ai-alignment/_map]]

domains/ai-alignment/interpretability-compute-costs-amplify-the-alignment-tax-through-massive-resource-requirements.md

@@ -0,0 +1,50 @@
+---
+type: claim
+domain: ai-alignment
+description: "Interpreting frontier models requires 20 petabytes of storage and GPT-3-level compute making interpretability prohibitively expensive for competitive deployment"
+confidence: likely
+source: "Gemma 2 interpretability requirements (2026 status report), Google DeepMind strategic pivot"
+created: 2026-01-01
+last_evaluated: 2026-01-01
+depends_on:
+  - "the alignment tax creates a structural race to the bottom because safety training costs capability and rational competitors skip it"
+---
+
+# Interpretability compute costs amplify the alignment tax through massive resource requirements
+
+Mechanistic interpretability of frontier models requires computational resources comparable to training the models themselves, creating a structural barrier to adoption in competitive deployment. Interpreting Gemma 2 required 20 petabytes of storage and GPT-3-level compute — costs that rational competitors will skip when interpretability is optional.
+
+This represents a specific mechanism by which the alignment tax operates. The alignment tax is not just the performance degradation from safety training (SAE reconstructions cause 10-40% performance loss), but also the massive infrastructure costs required to verify safety properties. Google DeepMind's Gemma Scope 2, despite being the largest open-source interpretability infrastructure (270M to 27B parameters), demonstrated that even well-resourced labs face prohibitive costs.
+
+When these costs are combined with the practical utility gap (simple baselines outperforming sophisticated interpretability on safety tasks), the economic case for interpretability in competitive deployment becomes structurally weak. DeepMind's strategic pivot away from fundamental SAE research despite building the largest infrastructure suggests that cost-benefit analysis favored cheaper approaches.
+
+## Evidence
+
+**Compute requirements:**
+- Interpreting Gemma 2 required 20 petabytes of storage and GPT-3-level compute (2026)
+- SAEs scaled to GPT-4 with 16 million latent variables
+- Circuit discovery for 25% of prompts required hours of human effort per analysis
+
+**Performance costs:**
+- SAE reconstructions cause 10-40% performance degradation on downstream tasks
+- This degradation is in addition to the compute costs, creating a double tax
+
+**Market response:**
+- Google DeepMind pivoted away from fundamental SAE research despite building the largest infrastructure
+- Strategic shift toward "pragmatic interpretability" suggests cost-benefit analysis favored cheaper approaches
+
+## Challenges
+
+Anthropic's integration of interpretability into Claude Sonnet 4.5 deployment decisions suggests that some organizations will pay the alignment tax when safety is a competitive differentiator. However, this may be viable only for labs where safety is a market positioning strategy, not for the broader competitive landscape.
+
+The compute costs may decrease as interpretability methods improve, but the fundamental tension remains: interpretability requires analyzing model internals at scale comparable to training, which is structurally expensive.
+
+---
+
+Relevant Notes:
+- [[the alignment tax creates a structural race to the bottom because safety training costs capability and rational competitors skip it]] — interpretability costs are a specific instantiation
+- [[voluntary safety pledges cannot survive competitive pressure because unilateral commitments are structurally punished when competitors advance without equivalent constraints]] — applies to interpretability adoption
+- [[safe AI development requires building alignment mechanisms before scaling capability]] — but interpretability costs make this economically difficult
+
+Topics:
+- [[domains/ai-alignment/_map]]

domains/ai-alignment/mechanistic-interpretability-diagnostic-capability-is-viable-but-comprehensive-alignment-vision-is-dead.md

@@ -0,0 +1,60 @@
+---
+type: claim
+domain: ai-alignment
+description: "The ambitious vision of achieving alignment through complete model understanding has been abandoned by leading labs in favor of pragmatic diagnostic approaches"
+confidence: likely
+source: "Neel Nanda statement, Google DeepMind strategic pivot (2026), Anthropic deployment integration"
+created: 2026-01-01
+last_evaluated: 2026-01-01
+depends_on:
+  - "scalable oversight degrades rapidly as capability gaps grow with debate achieving only 50 percent success at moderate gaps"
+  - "the alignment tax creates a structural race to the bottom because safety training costs capability and rational competitors skip it"
+secondary_domains: ["critical-systems"]
+---
+
+# Mechanistic interpretability diagnostic capability is viable but comprehensive alignment vision is dead
+
+The field of mechanistic interpretability has achieved genuine diagnostic capabilities while the original vision of comprehensive alignment through model understanding has been abandoned by leading research organizations. As Neel Nanda stated in 2026: "the most ambitious vision...is probably dead" but medium-risk approaches remain viable.
+
+Google DeepMind's strategic pivot away from fundamental SAE research represents the most significant signal. Despite building Gemma Scope 2 (the largest open-source interpretability infrastructure, spanning 270M to 27B parameter models), DeepMind found that SAEs **underperformed simple linear probes on practical safety tasks**. This led to an organizational shift toward "pragmatic interpretability" focused on task-specific utility rather than fundamental understanding.
+
+Meanwhile, Anthropic has demonstrated the viable middle ground: using mechanistic interpretability in the pre-deployment safety assessment of Claude Sonnet 4.5 marked the first integration of interpretability into production deployment decisions. Anthropic's stated goal is "reliably detecting most model problems by 2027" — a comprehensive diagnostic MRI approach rather than complete understanding.
+
+The practical utility gap remains the central tension: sophisticated interpretability methods are being outperformed by simple baseline approaches on safety-relevant detection tasks, even as the field makes genuine progress on specific diagnostic capabilities.
+
+## Evidence
+
+**Strategic divergence:**
+- Google DeepMind pivoted to "pragmatic interpretability" after finding SAEs underperformed linear probes on safety tasks (2026)
+- Anthropic targets "reliably detecting most model problems by 2027" — diagnostic scope, not comprehensive understanding
+- Neel Nanda's assessment: ambitious vision "probably dead," medium-risk approaches viable
+
+**Genuine diagnostic progress:**
+- Anthropic used mechanistic interpretability in pre-deployment safety assessment of Claude Sonnet 4.5 (first production integration)
+- Attribution graphs (Anthropic, March 2025) trace computational paths for ~25% of prompts
+- OpenAI identified "misaligned persona" features detectable via SAEs
+- Fine-tuning misalignment reversible with ~100 corrective training samples
+
+**Structural limitations preventing comprehensive understanding:**
+- SAE reconstructions cause 10-40% performance degradation on downstream tasks
+- No rigorous definition of "feature" exists
+- Deep networks exhibit "chaotic dynamics" where steering vectors become unpredictable after O(log(1/ε)) layers
+- Many circuit-finding queries proven NP-hard and inapproximable
+- Circuit discovery for 25% of prompts required hours of human effort per analysis
+
+## Challenges
+
+The practical utility gap challenges the value proposition: if simple baselines outperform sophisticated methods, why invest in interpretability? The answer appears to be that interpretability provides **legible explanations** and **mechanistic understanding** that baselines cannot, even when baselines achieve better detection rates. This matters for deployment decisions where stakeholders need to understand why a model was flagged, not just that it was flagged.
+
+Anthropic's production deployment suggests that legibility has independent value beyond detection rates, though this may only be economically viable for labs where safety is a market differentiator.
+
+---
+
+Relevant Notes:
+- [[scalable oversight degrades rapidly as capability gaps grow with debate achieving only 50 percent success at moderate gaps]] — interpretability faces similar scaling challenges
+- [[the alignment tax creates a structural race to the bottom because safety training costs capability and rational competitors skip it]] — interpretability's compute costs amplify this dynamic
+- [[AI alignment is a coordination problem not a technical problem]] — interpretability progress is real but bounded to technical diagnostics
+- [[safe AI development requires building alignment mechanisms before scaling capability]] — diagnostic interpretability enables this
+
+Topics:
+- [[domains/ai-alignment/_map]]

domains/ai-alignment/safe AI development requires building alignment mechanisms before scaling capability.md

@@ -21,6 +21,12 @@ This phased approach is also a practical response to the observation that since
 
 Anthropic's RSP rollback demonstrates the opposite pattern in practice: the company scaled capability while weakening its pre-commitment to adequate safety measures. The original RSP required guaranteeing safety measures were adequate *before* training new systems. The rollback removes this forcing function, allowing capability development to proceed with safety work repositioned as aspirational ('we hope to create a forcing function') rather than mandatory. This provides empirical evidence that even safety-focused organizations prioritize capability scaling over alignment-first development when competitive pressure intensifies, suggesting the claim may be normatively correct but descriptively violated by actual frontier labs under market conditions.
 
+
+### Additional Evidence (extend)
+*Source: [[2026-01-00-mechanistic-interpretability-2026-status-report]] | Added: 2026-03-11 | Extractor: anthropic/claude-sonnet-4.5*
+
+Anthropic's integration of mechanistic interpretability into Claude Sonnet 4.5 pre-deployment safety assessment (2026) demonstrates that interpretability can serve as an alignment mechanism built before deployment. However, the massive compute costs (20 petabytes, GPT-3-level compute) and performance degradation (10-40%) create economic barriers to widespread adoption. This suggests that 'building alignment mechanisms before scaling' is technically viable but economically difficult in competitive markets. The fact that only Anthropic has integrated interpretability into production deployment (as of 2026) despite years of research suggests that the alignment tax is prohibitive for most competitors, creating a structural barrier to pre-deployment safety mechanisms.
+
 ---
 
 Relevant Notes:

domains/ai-alignment/simple-linear-probes-outperform-SAEs-on-practical-safety-tasks-creating-a-utility-gap.md

@@ -0,0 +1,53 @@
+---
+type: claim
+domain: ai-alignment
+description: "Google DeepMind found that basic linear probes achieved better safety-relevant detection than sophisticated SAE-based interpretability methods"
+confidence: likely
+source: "Google DeepMind findings leading to strategic pivot (2026 status report)"
+created: 2026-01-01
+last_evaluated: 2026-01-01
+challenged_by:
+  - "Anthropic's production deployment use suggests SAEs provide unique value beyond detection rates"
+---
+
+# Simple linear probes outperform SAEs on practical safety tasks creating a utility gap
+
+Google DeepMind discovered that simple linear probes — basic machine learning methods that find linear relationships in model activations — outperformed sophisticated sparse autoencoder (SAE) approaches on practical safety-relevant detection tasks. This finding was significant enough to trigger a strategic pivot away from fundamental SAE research toward "pragmatic interpretability."
+
+This creates a central tension in interpretability research: the most theoretically sophisticated methods (SAEs with millions of latent variables, attribution graphs, circuit discovery) are being outperformed by simple baselines on the tasks that matter most for deployment safety. The practical utility gap suggests that interpretability's value may lie in **legible explanations** rather than **superior detection**, but this raises questions about whether the massive compute costs are justified.
+
+The finding is particularly striking because Google DeepMind built the largest open-source interpretability infrastructure (Gemma Scope 2, spanning 270M to 27B parameters) before reaching this conclusion. This suggests the result is robust across model scales and architectures, not an artifact of limited testing.
+
+## Evidence
+
+**DeepMind findings:**
+- SAEs underperformed simple linear probes on practical safety tasks (2026)
+- Finding led to strategic pivot toward "pragmatic interpretability" and away from fundamental SAE research
+- Occurred despite building Gemma Scope 2, the largest open-source interpretability infrastructure
+
+**Broader context:**
+- SAE reconstructions cause 10-40% performance degradation while providing inferior detection
+- Interpreting models requires 20 petabytes of storage and GPT-3-level compute
+- The combination (high cost + performance degradation + inferior detection) makes SAEs economically unviable for competitive deployment
+
+**Counter-evidence:**
+- Anthropic integrated interpretability into Claude Sonnet 4.5 deployment decisions (first production use)
+- Suggests interpretability provides value beyond detection rates — likely legible explanations and mechanistic understanding
+- OpenAI identified "misaligned persona" features via SAEs, suggesting some safety-relevant patterns require interpretability
+
+## Challenges
+
+The utility gap may be task-specific: linear probes may excel at detecting known safety issues (adversarial inputs, jailbreaks) while interpretability excels at discovering novel failure modes. If true, interpretability and baselines are complementary rather than competing approaches.
+
+Additionally, Anthropic's production deployment suggests that **legibility** has independent value: stakeholders may need to understand **why** a model was flagged, not just that it was flagged. Linear probes provide detection without explanation; interpretability provides both.
+
+---
+
+Relevant Notes:
+- [[mechanistic interpretability diagnostic capability is viable but comprehensive alignment vision is dead]] — the utility gap is central to this assessment
+- [[SAE reconstructions degrade model performance by 10 to 40 percent making interpretability costly at deployment]] — performance cost compounds the utility gap
+- [[interpretability compute costs amplify the alignment tax through massive resource requirements]] — compute costs compound the utility gap
+- [[scalable oversight degrades rapidly as capability gaps grow with debate achieving only 50 percent success at moderate gaps]] — similar pattern of sophisticated methods underperforming
+
+Topics:
+- [[domains/ai-alignment/_map]]

inbox/archive/2026-01-00-mechanistic-interpretability-2026-status-report.md

@@ -7,9 +7,15 @@ date: 2026-01-01
 domain: ai-alignment
 secondary_domains: []
 format: report
-status: unprocessed
+status: processed
 priority: high
 tags: [mechanistic-interpretability, SAE, safety, technical-alignment, limitations, DeepMind-pivot]
+processed_by: theseus
+processed_date: 2026-01-01
+claims_extracted: ["mechanistic-interpretability-diagnostic-capability-is-viable-but-comprehensive-alignment-vision-is-dead.md", "interpretability-compute-costs-amplify-the-alignment-tax-through-massive-resource-requirements.md", "SAE-reconstructions-degrade-model-performance-by-10-to-40-percent-making-interpretability-costly-at-deployment.md", "simple-linear-probes-outperform-SAEs-on-practical-safety-tasks-creating-a-utility-gap.md"]
+enrichments_applied: ["AI alignment is a coordination problem not a technical problem.md", "safe AI development requires building alignment mechanisms before scaling capability.md"]
+extraction_model: "anthropic/claude-sonnet-4.5"
+extraction_notes: "Four new claims extracted focusing on the diagnostic-vs-comprehensive framing, the practical utility gap (baselines outperforming SAEs), compute costs as alignment tax amplifier, and performance degradation. Four enrichments applied to existing alignment claims. The source directly tests the 'alignment is coordination not technical' thesis — interpretability is making real but bounded progress on diagnostics, cannot address preference diversity or coordination, and faces economic barriers (alignment tax) that create race-to-the-bottom dynamics. The DeepMind strategic pivot is the strongest signal: the leading interpretability lab deprioritizing its core technique because it underperforms baselines."
 ---
 
 ## Content
@@ -64,3 +70,13 @@ Comprehensive status report on mechanistic interpretability as of early 2026:
 PRIMARY CONNECTION: [[scalable oversight degrades rapidly as capability gaps grow with debate achieving only 50 percent success at moderate gaps]]
 WHY ARCHIVED: Provides 2026 status evidence on whether technical alignment (interpretability) can close the alignment gap — answer is "useful but bounded"
 EXTRACTION HINT: Focus on the practical utility gap (baselines outperform SAEs on safety tasks), the DeepMind strategic pivot, and Anthropic's production deployment use. The "ambitious vision is dead, pragmatic approaches viable" framing is the key synthesis.
+
+
+## Key Facts
+- MIT Technology Review named mechanistic interpretability a '2026 breakthrough technology'
+- January 2025 consensus paper by 29 researchers across 18 organizations established core open problems
+- Google DeepMind's Gemma Scope 2 (Dec 2025): 270M to 27B parameter models
+- SAEs scaled to GPT-4 with 16 million latent variables
+- Stream algorithm (Oct 2025): eliminates 97-99% of token interactions for near-linear time attention analysis
+- Fine-tuning misalignment reversible with ~100 corrective training samples
+- Attribution graphs trace computational paths for ~25% of prompts (Anthropic, March 2025)