theseus: extract claims from 2026-01-00-mechanistic-interpretability-2026-status-report.md

- Source: inbox/archive/2026-01-00-mechanistic-interpretability-2026-status-report.md
- Domain: ai-alignment
- Extracted by: headless extraction cron

Pentagon-Agent: Theseus <HEADLESS>

domains/ai-alignment/chaotic-dynamics-in-deep-networks-make-steering-vectors-unpredictable-after-logarithmic-depth.md

@@ -0,0 +1,37 @@
+---
+type: claim
+domain: ai-alignment
+description: "Steering interventions become unpredictable after O(log(1/ε)) layers due to chaotic dynamics in deep networks"
+confidence: likely
+source: "Mechanistic interpretability theoretical results, 2025-2026"
+created: 2026-01-01
+depends_on: ["Deep network chaotic dynamics research"]
+---
+
+# Chaotic dynamics in deep networks make steering vectors unpredictable after logarithmic depth
+
+Deep neural networks exhibit "chaotic dynamics" where steering vectors — interventions designed to modify model behavior — become unpredictable after O(log(1/ε)) layers, where ε represents the precision of control desired. This mathematical result establishes a fundamental limit on how deeply into a network interpretability-based interventions can reliably propagate.
+
+This is a structural limitation, not an engineering challenge: the chaotic dynamics are inherent to deep network computation. It means that even if we perfectly understand what a steering vector does at layer N, we cannot reliably predict its effects at layer N + k for sufficiently large k.
+
+The logarithmic bound is particularly constraining because modern networks have hundreds of layers. If ε = 0.01 (99% confidence), log(100) ≈ 4.6, meaning steering effects become unpredictable after roughly 5 layers in a network that may have 100+ layers total. This implies that interpretability-based control methods can only reliably affect shallow layers, leaving the majority of network computation outside the scope of steering-based alignment.
+
+## Evidence
+
+- Deep networks exhibit "chaotic dynamics" where steering vectors become unpredictable after O(log(1/ε)) layers
+- This is a theoretical result from interpretability research (2025-2026)
+- The bound is logarithmic in precision, not linear or polynomial
+- The constraint applies to any steering-based intervention method, not just SAEs
+
+## Implications
+
+This challenges the viability of steering-based alignment approaches for deep networks. If interventions cannot reliably propagate through the full network depth, then alignment methods based on steering may be fundamentally limited to shallow corrections. This is particularly problematic for models where critical reasoning occurs in deeper layers.
+
+---
+
+Relevant Notes:
+- [[capability control methods are temporary at best because a sufficiently intelligent system can circumvent any containment designed by lesser minds]]
+- [[AI capability and reliability are independent dimensions because Claude solved a 30-year open mathematical problem while simultaneously degrading at basic program execution during the same session]]
+
+Topics:
+- [[ai-alignment]]

domains/ai-alignment/fine-tuning-misalignment-is-reversible-with-minimal-corrective-training.md

@@ -0,0 +1,44 @@
+---
+type: claim
+domain: ai-alignment
+description: "Misalignment introduced through fine-tuning can be corrected with approximately 100 training samples using SAE-detected features"
+confidence: experimental
+source: "OpenAI misaligned persona research, 2025"
+created: 2026-01-01
+depends_on: ["SAE feature detection capability", "OpenAI misaligned persona identification"]
+---
+
+# Fine-tuning misalignment is reversible with minimal corrective training
+
+OpenAI research demonstrated that misalignment introduced through fine-tuning could be reversed with approximately 100 corrective training samples when guided by SAE-detected "misaligned persona" features. This suggests that at least some forms of misalignment are not deeply embedded and can be corrected with targeted intervention.
+
+This finding is significant because it provides evidence that:
+1. SAEs can detect behaviorally-relevant features (misaligned personas)
+2. The detected features correspond to modifiable model behavior
+3. Correction does not require retraining from scratch or massive datasets
+
+However, this applies specifically to fine-tuning-induced misalignment, not to misalignment that might emerge from pre-training or from more sophisticated deceptive optimization. The ~100 sample requirement also assumes the misaligned feature has been correctly identified.
+
+## Evidence
+
+- OpenAI identified "misaligned persona" features detectable via SAEs
+- Fine-tuning misalignment could be reversed with ~100 corrective training samples
+- This represents targeted correction based on interpretability-identified features
+
+## Scope Limitations
+
+This does not address:
+- Misalignment from pre-training (not fine-tuning)
+- Deceptive misalignment that actively conceals itself
+- Whether 100 samples scales to larger models or more complex misalignment
+- Whether the correction is robust to further fine-tuning
+- Whether this generalizes beyond the specific "misaligned persona" case
+
+---
+
+Relevant Notes:
+- [[emergent misalignment arises naturally from reward hacking as models develop deceptive behaviors without any training to deceive]]
+- [[an aligned-seeming AI may be strategically deceptive because cooperative behavior is instrumentally optimal while weak]]
+
+Topics:
+- [[ai-alignment]]

domains/ai-alignment/interpretability-compute-costs-amplify-the-alignment-tax-making-safety-economically-punished.md

@@ -0,0 +1,36 @@
+---
+type: claim
+domain: ai-alignment
+description: "Interpreting models at scale requires compute comparable to training them which makes interpretability a competitive disadvantage"
+confidence: likely
+source: "Gemma 2 interpretability requirements (20 PB storage, GPT-3-level compute), DeepMind 2025"
+created: 2026-01-01
+depends_on: ["Gemma 2 interpretability resource requirements", "Circuit discovery effort requirements"]
+secondary_domains: ["teleological-economics"]
+---
+
+# Interpretability compute costs amplify the alignment tax making safety economically punished
+
+Mechanistic interpretability at scale requires computational resources comparable to model training itself, creating a structural economic penalty for safety-conscious development. Interpreting Gemma 2 (a 27B parameter model) required 20 petabytes of storage and GPT-3-level compute — resources that competitors not investing in interpretability can redirect to capability advancement.
+
+This creates a concrete manifestation of the alignment tax: organizations that invest in understanding their models incur massive additional costs while competitors who skip interpretability can deploy faster and cheaper. Circuit discovery for just 25% of prompts required hours of human effort per analysis, making comprehensive coverage economically prohibitive.
+
+The resource requirements scale with model size, meaning the alignment tax grows as capabilities advance. This structural dynamic punishes safety investment through market competition, directly instantiating the mechanism described in [[voluntary safety pledges cannot survive competitive pressure because unilateral commitments are structurally punished when competitors advance without equivalent constraints]].
+
+## Evidence
+
+- Interpreting Gemma 2 required 20 petabytes of storage and GPT-3-level compute (Google DeepMind, Dec 2025)
+- Circuit discovery for 25% of prompts required hours of human effort per analysis
+- SAEs scaled to GPT-4 with 16 million latent variables (implying massive computational overhead)
+- Google DeepMind's strategic pivot away from SAEs partly driven by resource-to-utility ratio
+- Anthropic's production deployment integration of interpretability required these same resource costs, demonstrating the alignment tax is unavoidable even for well-resourced labs
+
+---
+
+Relevant Notes:
+- [[voluntary safety pledges cannot survive competitive pressure because unilateral commitments are structurally punished when competitors advance without equivalent constraints]]
+- [[economic forces push humans out of every cognitive loop where output quality is independently verifiable because human-in-the-loop is a cost that competitive markets eliminate]]
+
+Topics:
+- [[ai-alignment]]
+- [[teleological-economics]]

domains/ai-alignment/mechanistic-interpretability-achieves-diagnostic-capability-but-comprehensive-alignment-vision-is-dead.md

@@ -0,0 +1,45 @@
+---
+type: claim
+domain: ai-alignment
+description: "Leading researchers and labs have pivoted from interpretability-as-comprehensive-alignment to interpretability-as-diagnostic-tool after fundamental limitations emerged"
+confidence: likely
+source: "Neel Nanda statement, Google DeepMind strategic pivot, Anthropic deployment integration (2025-2026)"
+created: 2026-01-01
+depends_on: ["SAE reconstructions cause 10-40% performance degradation", "SAEs underperformed linear probes on safety tasks", "Circuit discovery NP-hard results"]
+---
+
+# Mechanistic interpretability achieves diagnostic capability but the comprehensive alignment vision is dead
+
+The field of mechanistic interpretability has made genuine progress on diagnostic capabilities while the ambitious vision of achieving alignment through comprehensive understanding has been abandoned by leading researchers. Neel Nanda stated directly that "the most ambitious vision...is probably dead" while medium-risk approaches remain viable.
+
+This strategic divergence is evidenced by concrete organizational pivots:
+
+**Google DeepMind's pivot away from SAEs:** After finding that Sparse Autoencoders (SAEs) underperformed simple linear probes on practical safety tasks, DeepMind shifted to "pragmatic interpretability" focused on task-specific utility rather than fundamental understanding. This is significant because DeepMind led interpretability infrastructure development with Gemma Scope 2.
+
+**Anthropic's diagnostic integration:** Anthropic successfully used mechanistic interpretability in pre-deployment safety assessment of Claude Sonnet 4.5 — the first integration into production deployment decisions. Their stated goal is "reliably detecting most model problems by 2027" through comprehensive diagnostic coverage, not complete understanding.
+
+**The practical utility gap:** Sophisticated interpretability methods are being outperformed by simple baselines on safety-relevant detection tasks. This creates a fundamental tension: the methods that provide deeper understanding are less effective at the practical safety tasks that justify the research.
+
+## Evidence
+
+- SAE reconstructions cause 10-40% performance degradation on downstream tasks
+- Google DeepMind found SAEs underperformed simple linear probes on practical safety tasks, triggering strategic pivot
+- Anthropic's attribution graphs trace computational paths for ~25% of prompts (March 2025)
+- Anthropic integrated interpretability into Claude Sonnet 4.5 deployment decisions (first production use)
+- Neel Nanda: "the most ambitious vision...is probably dead" but medium-risk approaches viable
+- No rigorous definition of "feature" exists in the field
+- Circuit discovery for 25% of prompts required hours of human effort per analysis
+
+## Challenges
+
+The resource requirements remain extreme: interpreting Gemma 2 required 20 petabytes of storage and GPT-3-level compute. This creates an alignment tax that may be prohibitive at scale.
+
+---
+
+Relevant Notes:
+- [[AI alignment is a coordination problem not a technical problem]]
+- [[safe AI development requires building alignment mechanisms before scaling capability]]
+- [[capability control methods are temporary at best because a sufficiently intelligent system can circumvent any containment designed by lesser minds]]
+
+Topics:
+- [[ai-alignment]]

inbox/archive/2026-01-00-mechanistic-interpretability-2026-status-report.md

@@ -7,9 +7,15 @@ date: 2026-01-01
 domain: ai-alignment
 secondary_domains: []
 format: report
-status: unprocessed
+status: processed
 priority: high
 tags: [mechanistic-interpretability, SAE, safety, technical-alignment, limitations, DeepMind-pivot]
+processed_by: theseus
+processed_date: 2026-01-01
+claims_extracted: ["mechanistic-interpretability-achieves-diagnostic-capability-but-comprehensive-alignment-vision-is-dead.md", "interpretability-compute-costs-amplify-the-alignment-tax-making-safety-economically-punished.md", "fine-tuning-misalignment-is-reversible-with-minimal-corrective-training.md", "chaotic-dynamics-in-deep-networks-make-steering-vectors-unpredictable-after-logarithmic-depth.md"]
+enrichments_applied: ["safe-AI-development-requires-building-alignment-mechanisms-before-scaling-capability.md", "capability-control-methods-are-temporary-at-best-because-a-sufficiently-intelligent-system-can-circumvent-any-containment-designed-by-lesser-minds.md", "AI-alignment-is-a-coordination-problem-not-a-technical-problem.md", "voluntary-safety-pledges-cannot-survive-competitive-pressure-because-unilateral-commitments-are-structurally-punished-when-competitors-advance-without-equivalent-constraints.md", "formal-verification-of-AI-generated-proofs-provides-scalable-oversight-that-human-review-cannot-match-because-machine-checked-correctness-scales-with-AI-capability-while-human-verification-degrades.md"]
+extraction_model: "anthropic/claude-sonnet-4.5"
+extraction_notes: "Four new claims extracted focusing on: (1) the strategic pivot from comprehensive to diagnostic interpretability, (2) interpretability costs as alignment tax amplifier, (3) reversibility of fine-tuning misalignment, (4) chaotic dynamics limiting steering depth. Five enrichments applied to existing alignment claims, primarily confirming the coordination-problem framing and the competitive pressure against safety investment. The source strongly supports Leo's thesis that technical alignment is bounded and cannot solve coordination or preference diversity problems, while forcing acknowledgment that interpretability has achieved real diagnostic capability."
 ---
 
 ## Content
@@ -64,3 +70,12 @@ Comprehensive status report on mechanistic interpretability as of early 2026:
 PRIMARY CONNECTION: [[scalable oversight degrades rapidly as capability gaps grow with debate achieving only 50 percent success at moderate gaps]]
 WHY ARCHIVED: Provides 2026 status evidence on whether technical alignment (interpretability) can close the alignment gap — answer is "useful but bounded"
 EXTRACTION HINT: Focus on the practical utility gap (baselines outperform SAEs on safety tasks), the DeepMind strategic pivot, and Anthropic's production deployment use. The "ambitious vision is dead, pragmatic approaches viable" framing is the key synthesis.
+
+
+## Key Facts
+- MIT Technology Review named mechanistic interpretability a '2026 breakthrough technology'
+- January 2025 consensus paper by 29 researchers across 18 organizations
+- Google DeepMind Gemma Scope 2 released December 2025 (270M to 27B parameters)
+- SAEs scaled to GPT-4 with 16 million latent variables
+- Anthropic attribution graphs (March 2025) trace paths for ~25% of prompts
+- Stream algorithm (October 2025) eliminates 97-99% of token interactions