---

type: claim
domain: ai-alignment
secondary_domains: [teleological-economics]
description: "Catalini et al. argue that AGI economics is governed by a Measurability Gap between what AI can execute and what humans can verify, creating pressure toward unverified deployment and a potential Hollow Economy"
confidence: likely
source: "Catalini, Hui & Wu, Some Simple Economics of AGI (arXiv 2602.20946, February 2026)"
created: 2026-03-16
supports:
- formal verification becomes economically necessary as AI-generated code scales because testing cannot detect adversarial overfitting and a proof cannot be gamed
reweave_edges:
- formal verification becomes economically necessary as AI-generated code scales because testing cannot detect adversarial overfitting and a proof cannot be gamed|supports|2026-03-28
sourced_from:
- inbox/archive/ai-alignment/2026-02-24-catalini-simple-economics-agi.md
---

# human verification bandwidth is the binding constraint on AGI economic impact not intelligence itself because the marginal cost of AI execution falls to zero while the capacity to validate audit and underwrite responsibility remains finite

Catalini et al. (2026) identify verification bandwidth — the human capacity to validate, audit, and underwrite responsibility for AI output — as the binding constraint on AGI's economic impact. As AI decouples cognition from biology, the marginal cost of measurable execution falls toward zero. But this creates a "Measurability Gap" between what systems can execute and what humans can practically oversee.

Two destabilizing forces emerge:

**The Missing Junior Loop.** AI collapses the apprenticeship pipeline. Junior roles traditionally served as both production AND training — the work was the learning. When AI handles junior-level production, the pipeline that produces senior judgment dries up. This creates a verification debt: the system needs more verification capacity (because AI output is growing) while simultaneously destroying the training ground that produces verifiers.

**The Codifier's Curse.** Domain experts who codify their knowledge into AI systems are codifying their own obsolescence. The rational individual response is to withhold knowledge — but the collective optimum requires sharing. This is a classic coordination failure that mirrors [[coordination failures arise from individually rational strategies that produce collectively irrational outcomes because the Nash equilibrium of non-cooperation dominates when trust and enforcement are absent]].

These pressures incentivize "unverified deployment" as economically rational, driving toward what Catalini calls a "Hollow Economy" — systems that execute at scale without adequate verification. The alternative — an "Augmented Economy" — requires deliberately scaling verification alongside capability.

This provides the economic mechanism for why [[scalable oversight degrades rapidly as capability gaps grow with debate achieving only 50 percent success at moderate gaps]]. Scalable oversight doesn't degrade because of some abstract capability gap — it degrades because verification is labor-intensive, labor is finite, and AI execution scales while verification doesn't. The economic framework makes the degradation curve predictable rather than mysterious.

For the Teleo collective: our multi-agent review pipeline is explicitly a verification scaling mechanism. The triage-first architecture proposal addresses exactly this bottleneck — don't spend verification bandwidth on sources unlikely to produce mergeable claims.


### Additional Evidence (extend)
*Source: [[2026-02-24-catalini-simple-economics-agi]] | Added: 2026-03-19*

Catalini et al. provide the full economic framework for why verification bandwidth is the constraint: they identify two competing cost curves (AI execution approaching zero vs. bounded human verification), two mechanisms that degrade verification over time (Missing Junior Loop and Codifier's Curse), and the economic incentive structure that makes unverified deployment rational at firm level. This extends the existing claim by showing not just that verification is the bottleneck, but WHY competitive markets systematically underinvest in it.

---

Relevant Notes:
- [[scalable oversight degrades rapidly as capability gaps grow with debate achieving only 50 percent success at moderate gaps]] — Catalini provides the economic mechanism for why oversight degrades
- [[coordination failures arise from individually rational strategies that produce collectively irrational outcomes because the Nash equilibrium of non-cooperation dominates when trust and enforcement are absent]] — the Codifier's Curse is a coordination failure
- [[economic forces push humans out of every cognitive loop where output quality is independently verifiable because human-in-the-loop is a cost that competitive markets eliminate]] — verification bandwidth constraint explains why markets push humans out
- [[formal verification of AI-generated proofs provides scalable oversight that human review cannot match because machine-checked correctness scales with AI capability while human verification degrades]] — formal verification is one solution to the verification bandwidth bottleneck
- [[single evaluator bottleneck means review throughput scales linearly with proposer count because one agent reviewing every PR caps collective output at the evaluators context window]] — our own pipeline exhibits this bottleneck

Topics:
- [[_map]]
