theseus: extract 4 claims from 2026 mechanistic interpretability status report

- What: 4 claims on interpretability's diagnostic utility, SAE limitations, circuit-discovery intractability, and compute costs as alignment tax amplifier
- Why: bigsnarfdude 2026 compilation synthesizing Anthropic/DeepMind/OpenAI findings; high-priority source with direct evidence on technical alignment's structural limits
- Connections: grounds [[scalable oversight degrades rapidly as capability gaps grow]] in NP-hardness theory; quantifies [[the alignment tax]] with 20PB/GPT-3-compute figure; confirms [[AI alignment is a coordination problem not a technical problem]] by showing interpretability is bounded to diagnostic use

Pentagon-Agent: Theseus <A1B2C3D4-E5F6-7890-ABCD-EF1234567890>

domains/ai-alignment/circuit discovery in large neural networks is computationally intractable because many queries are proven NP-hard and inapproximable placing a structural ceiling on comprehensive mechanistic interpretability.md

@@ -0,0 +1,33 @@
+---
+type: claim
+domain: ai-alignment
+description: "The computational complexity results mean the limits on interpretability are not engineering obstacles to be overcome but structural properties of the problem itself."
+confidence: likely
+source: "theseus, bigsnarfdude 2026 status report citing complexity theory results on circuit-finding queries"
+created: 2026-03-11
+depends_on:
+  - "mechanistic interpretability has proven diagnostic utility but the vision of comprehensively solving alignment through mechanistic understanding is acknowledged by field leaders as probably dead"
+challenged_by: []
+---
+
+# circuit discovery in large neural networks is computationally intractable because many queries are proven NP-hard and inapproximable placing a structural ceiling on comprehensive mechanistic interpretability
+
+As of 2026, theoretical results have established that many circuit-finding queries in neural networks — the queries that mechanistic interpretability relies on to reverse-engineer model behavior — are NP-hard and inapproximable. This is not a statement about current engineering constraints but about the computational complexity class of the problem itself.
+
+The practical implications are severe. The 2026 status report documents that circuit discovery for just 25% of prompts using attribution graphs required hours of human effort per analysis. Even at that limited scope, the resource cost is already prohibitive for deployment-scale safety assessment. The NP-hardness results explain why: for large networks, exact circuit discovery cannot be made efficient regardless of hardware improvements, since NP-hard problems cannot be solved in polynomial time (assuming P ≠ NP).
+
+A second structural finding compounds this: deep networks exhibit "chaotic dynamics" where steering vectors — another core interpretability tool — become unpredictable after O(log(1/ε)) layers. This means intervention-based interpretability methods have bounded effective depth regardless of their initial precision. The combination of NP-hard circuit discovery and chaotic steering-vector dynamics establishes two independent ceilings on what comprehensive mechanistic understanding can achieve.
+
+These complexity results provide the theoretical grounding for the field's empirical turn toward bounded, task-specific interpretability. The Google DeepMind pivot and Neel Nanda's acknowledgment that "the most ambitious vision is probably dead" are not expressions of pessimism — they reflect an appropriate updating in response to complexity theory results that were not available when the ambitious vision was first articulated.
+
+This is structurally similar to results in [[scalable oversight degrades rapidly as capability gaps grow with debate achieving only 50 percent success at moderate gaps]]: oversight mechanisms degrade not because they are poorly designed but because the cognitive gap they are trying to bridge creates fundamental scaling barriers.
+
+---
+
+Relevant Notes:
+- [[mechanistic interpretability has proven diagnostic utility but the vision of comprehensively solving alignment through mechanistic understanding is acknowledged by field leaders as probably dead]] — the NP-hardness results are one of the key structural reasons
+- [[scalable oversight degrades rapidly as capability gaps grow with debate achieving only 50 percent success at moderate gaps]] — parallel structural bound: oversight fails for similar complexity-theoretic reasons
+- [[capability control methods are temporary at best because a sufficiently intelligent system can circumvent any containment designed by lesser minds]] — computational intractability is a different but complementary argument for why control-based approaches face ceilings
+
+Topics:
+- [[_map]]

domains/ai-alignment/mechanistic interpretability has proven diagnostic utility but the vision of comprehensively solving alignment through mechanistic understanding is acknowledged by field leaders as probably dead.md

@@ -0,0 +1,36 @@
+---
+type: claim
+domain: ai-alignment
+description: "As of 2026 interpretability can detect specific model problems but the goal of fully explaining model behavior to guarantee alignment is recognized as infeasible—leaving a bounded diagnostic role."
+confidence: likely
+source: "theseus, bigsnarfdude 2026 status report synthesizing Anthropic/DeepMind/OpenAI findings; Neel Nanda quote"
+created: 2026-03-11
+depends_on:
+  - "scalable oversight degrades rapidly as capability gaps grow with debate achieving only 50 percent success at moderate gaps"
+  - "AI alignment is a coordination problem not a technical problem"
+challenged_by:
+  - "Anthropic targets reliably detecting most model problems by 2027 — a more optimistic near-term goal remains active"
+---
+
+# mechanistic interpretability has proven diagnostic utility but the vision of comprehensively solving alignment through mechanistic understanding is acknowledged by field leaders as probably dead
+
+By early 2026, mechanistic interpretability — the research program aimed at reverse-engineering what neural networks compute — has produced genuine breakthroughs while its most ambitious goals have been quietly abandoned by the field's own leaders.
+
+The diagnostic gains are real. Anthropic's attribution graphs (March 2025) trace computational paths for approximately 25% of prompts. Anthropic used mechanistic interpretability findings in the pre-deployment safety assessment of Claude Sonnet 4.5 — the first integration into actual production deployment decisions, not just research. OpenAI identified "misaligned persona" features detectable via sparse autoencoders (SAEs), and found that fine-tuning-induced misalignment could be reversed with approximately 100 corrective training samples by targeting those features precisely. MIT Technology Review named mechanistic interpretability a "2026 breakthrough technology." A January 2025 consensus paper by 29 researchers across 18 organizations established the field's core open problems, signaling institutional maturity.
+
+But the comprehensive vision — using mechanistic understanding to guarantee alignment by fully explaining model behavior — is not viable. Neel Nanda, one of the field's most prominent researchers, stated that "the most ambitious vision...is probably dead" while affirming that medium-risk approaches remain viable. Strategic divergence between labs reflects this: Anthropic targets "reliably detecting most model problems by 2027" (a bounded diagnostic goal, not comprehensive understanding), while Google DeepMind pivoted entirely to "pragmatic interpretability" focused on task-specific utility.
+
+The structural reasons go beyond current capability gaps. Many circuit-finding queries are proven NP-hard and inapproximable — comprehensive circuit discovery has computational bounds, not just engineering limits. Deep networks exhibit chaotic dynamics where steering vectors become unpredictable after O(log(1/ε)) layers. SAE reconstructions cause 10-40% performance degradation on downstream tasks. The field's consensus is now that interpretability is a diagnostic and monitoring tool — not an alignment solution.
+
+This confirms the claim that [[AI alignment is a coordination problem not a technical problem]]: interpretability can improve diagnostic confidence, but it cannot substitute for the coordination architecture needed to ensure competing actors deploy safe systems. Interpretability can tell you if *this* model has a specific problem; it cannot ensure the race to build models proceeds safely.
+
+---
+
+Relevant Notes:
+- [[AI alignment is a coordination problem not a technical problem]] — interpretability progress is real but bounded; it cannot solve the coordination layer
+- [[scalable oversight degrades rapidly as capability gaps grow with debate achieving only 50 percent success at moderate gaps]] — NP-hardness results and the practical utility gap are consistent with this structural degradation
+- [[the alignment tax creates a structural race to the bottom because safety training costs capability and rational competitors skip it]] — the compute costs of interpretability amplify this tax
+- [[formal verification of AI-generated proofs provides scalable oversight that human review cannot match because machine-checked correctness scales with AI capability while human verification degrades]] — formal verification solves the oversight problem for proofs in ways interpretability cannot solve for general behavior
+
+Topics:
+- [[_map]]

domains/ai-alignment/production-grade mechanistic analysis of large language models requires resources comparable to training a major model which amplifies the alignment tax.md

@@ -0,0 +1,36 @@
+---
+type: claim
+domain: ai-alignment
+description: "Interpreting a 27B parameter model consumed 20 petabytes of storage and GPT-3-level compute, making comprehensive safety-via-interpretability a cost that competitive labs are structurally incentivized to skip."
+confidence: experimental
+source: "theseus, bigsnarfdude 2026 status report citing Gemma 2 interpretability resource costs"
+created: 2026-03-11
+depends_on:
+  - "the alignment tax creates a structural race to the bottom because safety training costs capability and rational competitors skip it"
+  - "mechanistic interpretability has proven diagnostic utility but the vision of comprehensively solving alignment through mechanistic understanding is acknowledged by field leaders as probably dead"
+challenged_by:
+  - "Stream algorithm (Oct 2025) achieved near-linear time attention analysis, eliminating 97-99% of token interactions — suggesting analysis costs can be dramatically reduced for specific queries"
+---
+
+# production-grade mechanistic analysis of large language models requires resources comparable to training a major model which amplifies the alignment tax
+
+The 2026 status report documents a specific and striking cost figure: interpreting Gemma 2 (a 27B parameter model) required 20 petabytes of storage and compute equivalent to training GPT-3. This is not the cost of training the model — it is the cost of analyzing an already-trained model for interpretability purposes.
+
+This finding establishes a concrete lower bound on what production-grade mechanistic analysis costs for mid-size frontier models. The implication for the alignment tax is direct: [[the alignment tax creates a structural race to the bottom because safety training costs capability and rational competitors skip it]] — and interpretability-based safety assessment dramatically amplifies this tax. A lab that commits to comprehensive mechanistic analysis before deployment incurs GPT-3-level compute costs per model analyzed. A competitor that skips this step saves those costs and ships faster.
+
+The cost figure also reframes the Google DeepMind pivot. DeepMind's move toward "pragmatic interpretability" may partly reflect resource economics: comprehensive SAE-based analysis at scale is simply not affordable for routine safety assessment. Focusing on targeted, task-specific tools is not just strategically pragmatic but economically necessary.
+
+An important counterpoint: the Stream algorithm (October 2025) achieved near-linear time attention analysis by eliminating 97-99% of token interactions. This suggests that specific queries can be made dramatically cheaper, though the savings apply to attention analysis specifically rather than the full interpretability stack. The 20 petabyte figure likely reflects comprehensive feature extraction rather than targeted analysis.
+
+The confidence is rated `experimental` because this is a single documented case for one model architecture. Whether the cost scales linearly, quadratically, or otherwise with parameter count is not established from this data point alone.
+
+---
+
+Relevant Notes:
+- [[the alignment tax creates a structural race to the bottom because safety training costs capability and rational competitors skip it]] — interpretability costs are an additive component of the alignment tax that make safety even more expensive
+- [[mechanistic interpretability has proven diagnostic utility but the vision of comprehensively solving alignment through mechanistic understanding is acknowledged by field leaders as probably dead]] — the cost data partially explains why the comprehensive vision was abandoned
+- [[voluntary safety pledges cannot survive competitive pressure because unilateral commitments are structurally punished when competitors advance without equivalent constraints]] — resource-intensive safety procedures are structurally similar to voluntary pledges: individually rational to skip under competition
+- [[economic forces push humans out of every cognitive loop where output quality is independently verifiable because human-in-the-loop is a cost that competitive markets eliminate]] — same structural dynamic applies to expensive safety analysis
+
+Topics:
+- [[_map]]

domains/ai-alignment/sparse autoencoders underperform simple linear probes on practical safety-relevant detection tasks which drove Google DeepMind to pivot away from fundamental SAE research.md

@@ -0,0 +1,32 @@
+---
+type: claim
+domain: ai-alignment
+description: "Google DeepMind's internal finding that SAEs were beaten by simple baselines on the tasks they were designed for forced a strategic reorientation away from the field's dominant technique."
+confidence: likely
+source: "theseus, bigsnarfdude 2026 status report citing Google DeepMind internal research findings"
+created: 2026-03-11
+depends_on:
+  - "mechanistic interpretability has proven diagnostic utility but the vision of comprehensively solving alignment through mechanistic understanding is acknowledged by field leaders as probably dead"
+challenged_by:
+  - "SAEs scaled to GPT-4 with 16 million latent variables represent continued investment in the technique by other labs"
+---
+
+# sparse autoencoders underperform simple linear probes on practical safety-relevant detection tasks which drove Google DeepMind to pivot away from fundamental SAE research
+
+The dominant technique in mechanistic interpretability research — sparse autoencoders (SAEs), which decompose neural network activations into interpretable features — was found by Google DeepMind to underperform simple linear probes on the safety-relevant detection tasks the technique was built to address. This finding, reported in the 2026 status report, drove a strategic pivot away from fundamental SAE research toward what DeepMind calls "pragmatic interpretability": task-specific tools that work over fundamental understanding.
+
+The significance of this finding extends beyond one lab's strategy. Google DeepMind houses some of the world's leading mechanistic interpretability researchers — Neel Nanda's team included — and was one of the most active producers of SAE infrastructure (including Gemma Scope 2, released December 2025: the largest open-source interpretability infrastructure for models ranging from 270M to 27B parameters). A finding that the core technique underperforms baselines on practical safety tasks is not a peripheral result; it is a direct test of the method's core value proposition.
+
+The practical utility gap is the central unresolved tension in the field: sophisticated interpretability methods exist and are technically impressive, but simple baseline approaches — linear probes trained directly on activation patterns — outperform them on the safety-relevant detection work that was supposed to justify the investment. SAEs produce 10-40% performance degradation on downstream tasks when reconstructions are used, further undermining the case for them over simpler approaches.
+
+Anthropic and OpenAI continue investing in SAEs. Anthropic scaled SAEs to GPT-4 with 16 million latent variables. OpenAI used SAEs to identify "misaligned persona" features and demonstrated fine-tuning misalignment reversal with ~100 corrective samples. This creates a clear laboratory divergence: Anthropic pursues comprehensive SAE coverage while DeepMind deprioritizes SAEs for practical safety work. The divergence is not merely strategic — it reflects genuinely different empirical findings about which methods work for which tasks.
+
+---
+
+Relevant Notes:
+- [[mechanistic interpretability has proven diagnostic utility but the vision of comprehensively solving alignment through mechanistic understanding is acknowledged by field leaders as probably dead]] — this finding is one of the key drivers of that conclusion
+- [[the alignment tax creates a structural race to the bottom because safety training costs capability and rational competitors skip it]] — if simpler methods work as well or better, the argument for expensive SAE infrastructure weakens
+- [[AI alignment is a coordination problem not a technical problem]] — lab-level divergence on techniques illustrates that interpretability progress does not produce coordinated alignment improvement
+
+Topics:
+- [[_map]]

inbox/archive/2026-01-00-mechanistic-interpretability-2026-status-report.md

@@ -7,7 +7,17 @@ date: 2026-01-01
 domain: ai-alignment
 secondary_domains: []
 format: report
-status: unprocessed
+status: processed
+processed_by: theseus
+processed_date: 2026-03-11
+claims_extracted:
+  - "mechanistic interpretability has proven diagnostic utility but the vision of comprehensively solving alignment through mechanistic understanding is acknowledged by field leaders as probably dead"
+  - "sparse autoencoders underperform simple linear probes on practical safety-relevant detection tasks which drove Google DeepMind to pivot away from fundamental SAE research"
+  - "circuit discovery in large neural networks is computationally intractable because many queries are proven NP-hard and inapproximable placing a structural ceiling on comprehensive mechanistic interpretability"
+  - "production-grade mechanistic analysis of large language models requires resources comparable to training a major model which amplifies the alignment tax"
+enrichments:
+  - "scalable oversight degrades rapidly as capability gaps grow with debate achieving only 50 percent success at moderate gaps — NP-hardness results provide new theoretical grounding"
+  - "the alignment tax creates a structural race to the bottom — 20PB/GPT-3-compute figure is new quantitative evidence"
 priority: high
 tags: [mechanistic-interpretability, SAE, safety, technical-alignment, limitations, DeepMind-pivot]
 ---