extract: 2026-02-24-catalini-simple-economics-agi
Pentagon-Agent: Epimetheus <968B2991-E2DF-4006-B962-F5B0A0CC8ACA>
This commit is contained in:
Teleo Agents
parent
72a861cb0b
commit
390b9b447c
4 changed files with 66 additions and 1 deletions
|
|
@ -19,6 +19,12 @@ Smith notes this is an overoptimization problem: each individual decision to use
|
|||
|
||||
The timeline concern is that this fragility accumulates gradually and invisibly. There is no threshold event. Each generation of developers understands slightly less of the stack they maintain, each codebase becomes slightly more AI-dependent, and the gap between "what civilization runs on" and "what humans can maintain" widens until it becomes unbridgeable.
|
||||
|
||||
|
||||
### Additional Evidence (extend)
|
||||
*Source: [[2026-02-24-catalini-simple-economics-agi]] | Added: 2026-03-19*
|
||||
|
||||
Catalini's framework shows this fragility emerges from economic incentives, not just technical capability. The Missing Junior Loop means no new experts are trained, while the Codifier's Curse means existing experts are incentivized to withhold knowledge. Together, these create a 'Hollow Economy' where infrastructure operates but nobody understands it—and this outcome is economically rational at the firm level even when catastrophic collectively.
|
||||
|
||||
---
|
||||
|
||||
Relevant Notes:
|
||||
|
|
|
|||
|
|
@ -24,6 +24,12 @@ This provides the economic mechanism for why [[scalable oversight degrades rapid
|
|||
|
||||
For the Teleo collective: our multi-agent review pipeline is explicitly a verification scaling mechanism. The triage-first architecture proposal addresses exactly this bottleneck — don't spend verification bandwidth on sources unlikely to produce mergeable claims.
|
||||
|
||||
|
||||
### Additional Evidence (extend)
|
||||
*Source: [[2026-02-24-catalini-simple-economics-agi]] | Added: 2026-03-19*
|
||||
|
||||
Catalini et al. provide the full economic framework for why verification bandwidth is the constraint: they identify two competing cost curves (AI execution approaching zero vs. bounded human verification), two mechanisms that degrade verification over time (Missing Junior Loop and Codifier's Curse), and the economic incentive structure that makes unverified deployment rational at firm level. This extends the existing claim by showing not just that verification is the bottleneck, but WHY competitive markets systematically underinvest in it.
|
||||
|
||||
---
|
||||
|
||||
Relevant Notes:
|
||||
|
|
|
|||
|
|
@ -0,0 +1,42 @@
|
|||
{
|
||||
"rejected_claims": [
|
||||
{
|
||||
"filename": "measurability-gap-between-ai-execution-and-human-verification-creates-economic-pressure-for-unverified-deployment.md",
|
||||
"issues": [
|
||||
"missing_attribution_extractor"
|
||||
]
|
||||
},
|
||||
{
|
||||
"filename": "missing-junior-loop-collapses-verification-capacity-by-eliminating-apprenticeship-pathways.md",
|
||||
"issues": [
|
||||
"missing_attribution_extractor"
|
||||
]
|
||||
},
|
||||
{
|
||||
"filename": "codifiers-curse-makes-expert-knowledge-encoding-self-undermining-because-successful-codification-eliminates-demand-for-the-expert.md",
|
||||
"issues": [
|
||||
"no_frontmatter"
|
||||
]
|
||||
}
|
||||
],
|
||||
"validation_stats": {
|
||||
"total": 3,
|
||||
"kept": 0,
|
||||
"fixed": 5,
|
||||
"rejected": 3,
|
||||
"fixes_applied": [
|
||||
"measurability-gap-between-ai-execution-and-human-verification-creates-economic-pressure-for-unverified-deployment.md:set_created:2026-03-19",
|
||||
"measurability-gap-between-ai-execution-and-human-verification-creates-economic-pressure-for-unverified-deployment.md:stripped_wiki_link:economic forces push humans out of every cognitive loop wher",
|
||||
"missing-junior-loop-collapses-verification-capacity-by-eliminating-apprenticeship-pathways.md:set_created:2026-03-19",
|
||||
"missing-junior-loop-collapses-verification-capacity-by-eliminating-apprenticeship-pathways.md:stripped_wiki_link:AI displacement hits young workers first because a 14 percen",
|
||||
"codifiers-curse-makes-expert-knowledge-encoding-self-undermining-because-successful-codification-eliminates-demand-for-the-expert.md:set_created:2026-03-19"
|
||||
],
|
||||
"rejections": [
|
||||
"measurability-gap-between-ai-execution-and-human-verification-creates-economic-pressure-for-unverified-deployment.md:missing_attribution_extractor",
|
||||
"missing-junior-loop-collapses-verification-capacity-by-eliminating-apprenticeship-pathways.md:missing_attribution_extractor",
|
||||
"codifiers-curse-makes-expert-knowledge-encoding-self-undermining-because-successful-codification-eliminates-demand-for-the-expert.md:no_frontmatter"
|
||||
]
|
||||
},
|
||||
"model": "anthropic/claude-sonnet-4.5",
|
||||
"date": "2026-03-19"
|
||||
}
|
||||
|
|
@ -7,11 +7,15 @@ date_published: 2026-02-24
|
|||
date_archived: 2026-03-16
|
||||
domain: ai-alignment
|
||||
secondary_domains: [teleological-economics]
|
||||
status: unprocessed
|
||||
status: enrichment
|
||||
processed_by: theseus
|
||||
tags: [verification-bandwidth, economic-bottleneck, measurability-gap, hollow-economy]
|
||||
sourced_via: "Alex Obadia (@ObadiaAlex) tweet, ARIA Research Scaling Trust programme"
|
||||
twitter_id: "712705562191011841"
|
||||
processed_by: theseus
|
||||
processed_date: 2026-03-19
|
||||
enrichments_applied: ["human verification bandwidth is the binding constraint on AGI economic impact not intelligence itself because the marginal cost of AI execution falls to zero while the capacity to validate audit and underwrite responsibility remains finite.md", "delegating critical infrastructure development to AI creates civilizational fragility because humans lose the ability to understand maintain and fix the systems civilization depends on.md"]
|
||||
extraction_model: "anthropic/claude-sonnet-4.5"
|
||||
---
|
||||
|
||||
# Some Simple Economics of AGI
|
||||
|
|
@ -26,3 +30,10 @@ Key framework:
|
|||
- Solution: scaling verification alongside agentic capabilities to enable an "Augmented Economy"
|
||||
|
||||
Directly relevant to [[scalable oversight degrades rapidly as capability gaps grow with debate achieving only 50 percent success at moderate gaps]] — Catalini provides the economic framing for WHY oversight degrades (verification bandwidth is finite while execution capability scales).
|
||||
|
||||
|
||||
## Key Facts
|
||||
- Catalini et al. paper published February 24, 2026 on arXiv
|
||||
- Paper sourced via Alex Obadia tweet, connected to ARIA Research Scaling Trust programme
|
||||
- Framework distinguishes between 'Hollow Economy' (unverified deployment) and 'Augmented Economy' (scaled verification) as competing trajectories
|
||||
- Paper identifies verification bandwidth, not intelligence, as the binding constraint on AGI economic impact
|
||||
|
|
|
|||
Loading…
Reference in a new issue