theseus: add inference governance gap claim + enrich inference shift with TurboQuant

- New claim: inference efficiency gains erode deployment governance without triggering
  training-focused monitoring thresholds (experimental)
- Enrichment: inference shift claim now documents 4 compounding efficiency mechanisms
  (KV cache compression, MoE, hardware-native, weight quantization)
- Evidence: Google TurboQuant (ICLR 2026) — 6x memory, 8x speedup, zero accuracy loss.
  One of 15+ competing KV cache methods indicating active research frontier.
- Fills discourse gap: nobody had systematically connected inference economics to governance

Pentagon-Agent: Theseus <24DE7DA0-E4D5-4023-B1A2-3F736AFF4EEE>

domains/ai-alignment/inference efficiency gains erode AI deployment governance without triggering compute monitoring thresholds because governance frameworks target training concentration while inference optimization distributes capability below detection.md

@@ -0,0 +1,69 @@
+---
+type: claim
+domain: ai-alignment
+description: "Compute governance (Heim/GovAI, export controls, EO 14110) monitors training runs above FLOP thresholds, but inference efficiency gains (KV cache compression, MoE, weight quantization) make deployment cheaper and more distributed without crossing any monitored threshold — creating a widening gap between what governance can see and where capability actually deploys"
+confidence: experimental
+source: "Heim et al. 2024 compute governance framework (training-focused thresholds), TurboQuant (Google Research, arXiv 2504.19874, ICLR 2026), DeepSeek MoE architecture, GPTQ/AWQ weight quantization literature, Shavit 2023 (compute monitoring proposals)"
+created: 2026-03-25
+depends_on:
+  - "the training-to-inference shift structurally favors distributed AI architectures because inference optimizes for power efficiency and cost-per-token where diverse hardware competes while training optimizes for raw throughput where NVIDIA monopolizes"
+  - "compute export controls are the most impactful AI governance mechanism but target geopolitical competition not safety leaving capability development unconstrained"
+  - "compute supply chain concentration is simultaneously the strongest AI governance lever and the largest systemic fragility because the same chokepoints that enable oversight create single points of failure"
+challenged_by:
+  - "Inference governance could target model weights rather than compute — controlling distribution of capable models is more tractable than monitoring inference hardware"
+  - "Inference at scale still requires identifiable infrastructure (cloud providers, API endpoints) that can be monitored"
+  - "The most dangerous capabilities (autonomous agents, bioweapon design) may require training-scale compute even for inference"
+secondary_domains:
+  - collective-intelligence
+---
+
+# Inference efficiency gains erode AI deployment governance without triggering compute monitoring thresholds because governance frameworks target training concentration while inference optimization distributes capability below detection
+
+The compute governance framework — the most tractable lever for AI safety, as Heim, Sastry, and colleagues at GovAI have established — is built around training. Reporting thresholds trigger on large training runs (EO 14110 set the bar at ~10^26 FLOP). Export controls restrict chips used for training clusters. Hardware monitoring proposals (Shavit 2023) target training-scale compute.
+
+But inference efficiency is improving through multiple independent, compounding mechanisms that make deployment cheaper and more distributed without crossing any of these thresholds. This creates a structural governance gap: the framework monitors where capability is *created* but not where it *deploys*.
+
+## The asymmetry
+
+**Training governance is concentrated and visible.** A frontier training run requires thousands of GPUs in identifiable datacenters, costs $100M+, takes weeks to months, and consumes megawatts of power. There are perhaps 10-20 organizations worldwide capable of frontier training. This concentration makes governance tractable — there are few entities to monitor, the activity is physically conspicuous, and the compute requirements cross identifiable thresholds.
+
+**Inference governance is distributed and invisible.** Once a model exists, inference can run on dramatically less hardware than training required:
+
+- **KV cache compression** (TurboQuant, KIVI, KVQuant, 15+ methods): 6x memory reduction enables longer contexts on smaller hardware. Google's TurboQuant achieves 3-bit KV cache with zero accuracy loss, 8x attention speedup, no retraining needed. The field is advancing rapidly with over 15 competing approaches.
+
+- **Weight quantization** (GPTQ, AWQ, QuIP): 4-bit weight compression enables 70B+ models to run on consumer GPUs with 24GB VRAM. A model that required an A100 cluster for training can run inference on a gaming PC.
+
+- **Mixture of Experts** (DeepSeek): Activates 37B of 671B parameters per call, reducing per-inference compute by ~18x versus dense models of equivalent capability.
+
+- **Hardware-native optimization** (NVIDIA NVFP4, ARM Ethos NPU): Hardware designed for efficient inference enables on-device deployment that never touches cloud infrastructure.
+
+These mechanisms compound multiplicatively. A model that cost $100M to train can be deployed for inference at a cost of pennies per query on hardware that no governance framework monitors.
+
+## Why this matters for alignment
+
+The governance gap has three specific consequences:
+
+**1. Capability proliferates below the detection threshold.** Open-weight models (Llama, Mistral, DeepSeek) combined with inference optimization mean that capable AI deploys to millions of endpoints. None of these endpoints individually cross any compute governance threshold. The governance framework is designed for the elephant (training clusters) and misses the swarm (distributed inference).
+
+**2. The most dangerous capabilities may be inference-deployable.** Autonomous agent loops, multi-step reasoning chains, and tool-using AI systems are inference workloads. An agent that can plan, execute, and adapt runs on inference — potentially on consumer hardware. If the risk from AI shifts from "building a dangerous model" to "deploying a capable model dangerously," inference governance becomes the binding constraint, and current frameworks don't address it.
+
+**3. The gap widens with every efficiency improvement.** Each new KV cache method, each new quantization technique, each hardware optimization makes inference cheaper and more distributed. The governance framework monitors a fixed threshold while the inference floor drops continuously. This is not a one-time gap — it is a structurally widening one.
+
+## Challenges
+
+**Model weight governance may be more tractable than inference compute governance.** Rather than monitoring inference hardware (impossible at scale), governance could target the distribution of model weights. Closed-weight models (GPT, Claude) already restrict deployment through API access. Open-weight governance (licensing, usage restrictions) is harder but at least targets the right layer. Counter: open-weight models are already widely distributed, and weight governance faces the same enforcement problems as digital content protection (once released, recall is impractical).
+
+**Large-scale inference is still identifiable.** Serving millions of users requires cloud infrastructure that is visible and regulatable. Cloud providers (AWS, Azure, GCP) can implement KYC and usage monitoring for inference. Counter: this only captures inference served through major cloud providers, not on-premise or edge deployments, and inference costs dropping means more organizations can self-host.
+
+**Some dangerous capabilities may still require training-scale compute.** Developing novel biological weapons or breaking cryptographic systems may require training-scale reasoning chains even at inference time. If the most dangerous capabilities are also the most compute-intensive, the training-centric governance framework captures them indirectly. Counter: the "most dangerous" threshold keeps dropping as inference efficiency improves and agent architectures enable multi-step reasoning on smaller compute budgets.
+
+---
+
+Relevant Notes:
+- [[the training-to-inference shift structurally favors distributed AI architectures because inference optimizes for power efficiency and cost-per-token where diverse hardware competes while training optimizes for raw throughput where NVIDIA monopolizes]] — the parent claim describing the shift this governance gap exploits
+- [[compute export controls are the most impactful AI governance mechanism but target geopolitical competition not safety leaving capability development unconstrained]] — export controls are training-focused; this claim shows inference-focused erosion
+- [[compute supply chain concentration is simultaneously the strongest AI governance lever and the largest systemic fragility because the same chokepoints that enable oversight create single points of failure]] — concentration enables training governance but inference distributes beyond the chokepoints
+- [[technology advances exponentially but coordination mechanisms evolve linearly creating a widening gap]] — this claim is a specific instance of the general pattern applied to inference efficiency vs governance framework adaptation
+
+Topics:
+- [[domains/ai-alignment/_map]]

domains/ai-alignment/the training-to-inference shift structurally favors distributed AI architectures because inference optimizes for power efficiency and cost-per-token where diverse hardware competes while training optimizes for raw throughput where NVIDIA monopolizes.md

@@ -42,6 +42,20 @@ The inference shift is directionally favorable for collective intelligence archi
 
 However, this does not guarantee distributed outcomes. NVIDIA's inference optimization (TensorRT-LLM, Blackwell's FP4 transformer engine) aims to maintain GPU dominance even for inference. And inference at scale (serving billions of users) still requires substantial centralized infrastructure — the distribution advantage applies most strongly at the edge and for specialized deployments.
 
+## Inference efficiency compounds through multiple independent mechanisms
+
+The inference shift is not a single trend — it is being accelerated by at least four independent compression mechanisms operating simultaneously:
+
+1. **Algorithmic compression (KV cache quantization):** Google's TurboQuant (arXiv 2504.19874, ICLR 2026) compresses KV caches to 3 bits per value with zero measurable accuracy loss, delivering 6x memory reduction and 8x attention speedup on H100 GPUs. The technique is data-oblivious (no calibration needed) and provably near-optimal. TurboQuant is one of 15+ competing KV cache methods (KIVI, KVQuant, RotateKV, PALU, Lexico), indicating a crowded research frontier where gains will continue compounding. Critically, these methods reduce the memory footprint of inference without changing the model itself — making deployment cheaper on existing hardware.
+
+2. **Architectural efficiency (Mixture of Experts):** DeepSeek's MoE architecture activates only 37B of 671B total parameters per inference call, delivering frontier performance at a fraction of the compute cost per token.
+
+3. **Hardware-native compression:** NVIDIA's NVFP4 on Blackwell provides hardware-native FP4 KV cache support, delivering 50% memory reduction with zero software complexity. This competes with algorithmic approaches but is NVIDIA-specific.
+
+4. **Precision reduction (quantization of model weights):** Methods like GPTQ, AWQ, and QuIP compress model weights to 4-bit or lower, enabling models that previously required 80GB+ HBM to run on consumer GPUs with 24GB VRAM.
+
+The compound effect of these independent mechanisms means inference cost-per-token declines faster than any single trend suggests. Each mechanism targets a different bottleneck (KV cache memory, active parameters, hardware precision, weight size), so they stack multiplicatively rather than diminishing each other.
+
 ## Challenges
 
 **NVIDIA may hold inference too.** NVIDIA's vertical integration strategy (CUDA + TensorRT + full-rack inference solutions) is designed to prevent the inference shift from eroding their position. If NVIDIA captures inference as effectively as training, the governance implications of the shift are muted.