extract: 2026-02-24-catalini-simple-economics-agi

Pentagon-Agent: Epimetheus <968B2991-E2DF-4006-B962-F5B0A0CC8ACA>

domains/ai-alignment/delegating critical infrastructure development to AI creates civilizational fragility because humans lose the ability to understand maintain and fix the systems civilization depends on.md

@@ -25,6 +25,12 @@ The timeline concern is that this fragility accumulates gradually and invisibly.
 
 Catalini's framework shows this fragility emerges from economic incentives, not just technical capability. The Missing Junior Loop means no new experts are trained, while the Codifier's Curse means existing experts are incentivized to withhold knowledge. Together, these create a 'Hollow Economy' where infrastructure operates but nobody understands it—and this outcome is economically rational at the firm level even when catastrophic collectively.
 
+
+### Additional Evidence (extend)
+*Source: [[2026-02-24-catalini-simple-economics-agi]] | Added: 2026-03-19*
+
+Catalini et al. formalize this as the 'Hollow Economy' trajectory where the Measurability Gap (execution capability exceeding verification capacity) combines with the Missing Junior Loop (apprenticeship collapse) to create systems that operate beyond human comprehension. The paper frames this as an economic attractor state, not just a technical risk—competitive pressure makes unverified deployment rational even when civilizationally catastrophic.
+
 ---
 
 Relevant Notes:

domains/ai-alignment/economic forces push humans out of every cognitive loop where output quality is independently verifiable because human-in-the-loop is a cost that competitive markets eliminate.md

@@ -19,6 +19,12 @@ The alignment implications are severe. Human-in-the-loop is the default safety a
 
 This creates a structural inversion: the market preserves human-in-the-loop exactly where it's least useful (unverifiable domains where humans can't easily evaluate AI output either) and removes it exactly where it's most useful (verifiable domains where bad outputs are detectable but only if someone is looking).
 
+
+### Additional Evidence (extend)
+*Source: [[2026-02-24-catalini-simple-economics-agi]] | Added: 2026-03-19*
+
+Catalini et al. add nuance: the pressure isn't just to remove humans from verifiable loops, but creates a structural incentive to deploy in domains where verification is expensive or impossible. The 'Measurability Gap' means systems can execute in domains where output quality is NOT independently verifiable, and competitive dynamics reward doing so anyway. This extends the mechanism from 'humans exit verifiable loops' to 'systems enter unverifiable domains.'
+
 ---
 
 Relevant Notes:

domains/ai-alignment/human verification bandwidth is the binding constraint on AGI economic impact not intelligence itself because the marginal cost of AI execution falls to zero while the capacity to validate audit and underwrite responsibility remains finite.md

@@ -30,6 +30,12 @@ For the Teleo collective: our multi-agent review pipeline is explicitly a verifi
 
 Catalini et al. provide the full economic framework for why verification bandwidth is the constraint: they identify two competing cost curves (AI execution approaching zero vs. bounded human verification), two mechanisms that degrade verification over time (Missing Junior Loop and Codifier's Curse), and the economic incentive structure that makes unverified deployment rational at firm level. This extends the existing claim by showing not just that verification is the bottleneck, but WHY competitive markets systematically underinvest in it.
 
+
+### Additional Evidence (extend)
+*Source: [[2026-02-24-catalini-simple-economics-agi]] | Added: 2026-03-19*
+
+Catalini et al. provide the economic mechanism for WHY verification bandwidth becomes binding: competitive markets create structural pressure for 'unverified deployment' because verification is expensive while execution approaches zero cost. This generates what they term the 'Measurability Gap' and drives toward a 'Hollow Economy' trajectory where systems operate beyond human oversight. The paper identifies two compounding forces: the 'Missing Junior Loop' (apprenticeship collapse) and 'Codifier's Curse' (experts eliminating their own future employment).
+
 ---
 
 Relevant Notes:

inbox/queue/.extraction-debug/2026-02-24-catalini-simple-economics-agi.json

@@ -3,38 +3,33 @@
     {
       "filename": "measurability-gap-between-ai-execution-and-human-verification-creates-economic-pressure-for-unverified-deployment.md",
       "issues": [
-        "missing_attribution_extractor"
+        "no_frontmatter"
       ]
     },
     {
-      "filename": "missing-junior-loop-collapses-verification-capacity-by-eliminating-apprenticeship-pathways.md",
-      "issues": [
-        "missing_attribution_extractor"
-      ]
-    },
-    {
-      "filename": "codifiers-curse-makes-expert-knowledge-encoding-self-undermining-because-successful-codification-eliminates-demand-for-the-expert.md",
+      "filename": "apprenticeship-collapse-eliminates-verification-capacity-pipeline-because-ai-handles-junior-work-that-trains-future-experts.md",
       "issues": [
         "no_frontmatter"
       ]
     }
   ],
   "validation_stats": {
-    "total": 3,
+    "total": 2,
     "kept": 0,
-    "fixed": 5,
-    "rejected": 3,
+    "fixed": 7,
+    "rejected": 2,
     "fixes_applied": [
       "measurability-gap-between-ai-execution-and-human-verification-creates-economic-pressure-for-unverified-deployment.md:set_created:2026-03-19",
+      "measurability-gap-between-ai-execution-and-human-verification-creates-economic-pressure-for-unverified-deployment.md:stripped_wiki_link:human verification bandwidth is the binding constraint on AG",
       "measurability-gap-between-ai-execution-and-human-verification-creates-economic-pressure-for-unverified-deployment.md:stripped_wiki_link:economic forces push humans out of every cognitive loop wher",
-      "missing-junior-loop-collapses-verification-capacity-by-eliminating-apprenticeship-pathways.md:set_created:2026-03-19",
-      "missing-junior-loop-collapses-verification-capacity-by-eliminating-apprenticeship-pathways.md:stripped_wiki_link:AI displacement hits young workers first because a 14 percen",
-      "codifiers-curse-makes-expert-knowledge-encoding-self-undermining-because-successful-codification-eliminates-demand-for-the-expert.md:set_created:2026-03-19"
+      "apprenticeship-collapse-eliminates-verification-capacity-pipeline-because-ai-handles-junior-work-that-trains-future-experts.md:set_created:2026-03-19",
+      "apprenticeship-collapse-eliminates-verification-capacity-pipeline-because-ai-handles-junior-work-that-trains-future-experts.md:stripped_wiki_link:delegating critical infrastructure development to AI creates",
+      "apprenticeship-collapse-eliminates-verification-capacity-pipeline-because-ai-handles-junior-work-that-trains-future-experts.md:stripped_wiki_link:AI displacement hits young workers first because a 14 percen",
+      "apprenticeship-collapse-eliminates-verification-capacity-pipeline-because-ai-handles-junior-work-that-trains-future-experts.md:stripped_wiki_link:metis is practical knowledge that can only be acquired throu"
     ],
     "rejections": [
-      "measurability-gap-between-ai-execution-and-human-verification-creates-economic-pressure-for-unverified-deployment.md:missing_attribution_extractor",
-      "missing-junior-loop-collapses-verification-capacity-by-eliminating-apprenticeship-pathways.md:missing_attribution_extractor",
-      "codifiers-curse-makes-expert-knowledge-encoding-self-undermining-because-successful-codification-eliminates-demand-for-the-expert.md:no_frontmatter"
+      "measurability-gap-between-ai-execution-and-human-verification-creates-economic-pressure-for-unverified-deployment.md:no_frontmatter",
+      "apprenticeship-collapse-eliminates-verification-capacity-pipeline-because-ai-handles-junior-work-that-trains-future-experts.md:no_frontmatter"
     ]
   },
   "model": "anthropic/claude-sonnet-4.5",

inbox/queue/2026-02-24-catalini-simple-economics-agi.md

@@ -16,6 +16,10 @@ processed_by: theseus
 processed_date: 2026-03-19
 enrichments_applied: ["human verification bandwidth is the binding constraint on AGI economic impact not intelligence itself because the marginal cost of AI execution falls to zero while the capacity to validate audit and underwrite responsibility remains finite.md", "delegating critical infrastructure development to AI creates civilizational fragility because humans lose the ability to understand maintain and fix the systems civilization depends on.md"]
 extraction_model: "anthropic/claude-sonnet-4.5"
+processed_by: theseus
+processed_date: 2026-03-19
+enrichments_applied: ["human verification bandwidth is the binding constraint on AGI economic impact not intelligence itself because the marginal cost of AI execution falls to zero while the capacity to validate audit and underwrite responsibility remains finite.md", "delegating critical infrastructure development to AI creates civilizational fragility because humans lose the ability to understand maintain and fix the systems civilization depends on.md", "economic forces push humans out of every cognitive loop where output quality is independently verifiable because human-in-the-loop is a cost that competitive markets eliminate.md"]
+extraction_model: "anthropic/claude-sonnet-4.5"
 ---
 
 # Some Simple Economics of AGI
@@ -37,3 +41,10 @@ Directly relevant to [[scalable oversight degrades rapidly as capability gaps gr
 - Paper sourced via Alex Obadia tweet, connected to ARIA Research Scaling Trust programme
 - Framework distinguishes between 'Hollow Economy' (unverified deployment) and 'Augmented Economy' (scaled verification) as competing trajectories
 - Paper identifies verification bandwidth, not intelligence, as the binding constraint on AGI economic impact
+
+
+## Key Facts
+- Catalini et al. paper 'Some Simple Economics of AGI' published February 24, 2026 on arXiv
+- Paper sourced via Alex Obadia (@ObadiaAlex) tweet, connected to ARIA Research Scaling Trust programme
+- Framework distinguishes between 'Hollow Economy' (unverified deployment) and 'Augmented Economy' (scaled verification) as competing trajectories
+- Paper identifies three key mechanisms: Measurability Gap, Missing Junior Loop, and Codifier's Curse