teleo/teleo-codex
8
0
Fork 0
Code Issues 22 Pull requests 39 Projects Releases Packages Wiki Activity Actions 1
teleo-codex/inbox/queue/2026-02-16-noahopinion-updated-thoughts-ai-risk.md
Teleo Agents 541766ac73 extract: 2026-02-16-noahopinion-updated-thoughts-ai-risk 
Pentagon-Agent: Epimetheus <3D35839A-7722-4740-B93D-51157F7D5E70>
2026-03-20 16:26:31 +00:00

2.3 KiB
Raw Blame History

title author source date processed_by processed_date type domain status claims_extracted processed_by processed_date extraction_model extraction_notes
Updated thoughts on AI risk Noah Smith Noahopinion (Substack) 2026-02-16 theseus 2026-03-06 newsletter ai-alignment null-result
economic forces push humans out of every cognitive loop where output quality is independently verifiable because human-in-the-loop is a cost that competitive markets eliminate
delegating critical infrastructure development to AI creates civilizational fragility because humans lose the ability to understand maintain and fix the systems civilization depends on
AI lowers the expertise barrier for engineering biological weapons from PhD-level to amateur which makes bioterrorism the most proximate AI-enabled existential risk
theseus 2026-03-20 anthropic/claude-sonnet-4.5 LLM returned 0 claims, 0 rejected by validator

title: "Updated thoughts on AI risk" author: Noah Smith source: Noahopinion (Substack) date: 2026-02-16 processed_by: theseus processed_date: 2026-03-06 type: newsletter domain: ai-alignment status: null-result claims_extracted:

  • "economic forces push humans out of every cognitive loop where output quality is independently verifiable because human-in-the-loop is a cost that competitive markets eliminate"
  • "delegating critical infrastructure development to AI creates civilizational fragility because humans lose the ability to understand maintain and fix the systems civilization depends on"
  • "AI lowers the expertise barrier for engineering biological weapons from PhD-level to amateur which makes bioterrorism the most proximate AI-enabled existential risk" processed_by: theseus processed_date: 2026-03-20 extraction_model: "anthropic/claude-sonnet-4.5" extraction_notes: "LLM returned 0 claims, 0 rejected by validator"

Updated thoughts on AI risk

Noah Smith's shift from 2023 AI optimism to increased concern about existential risk. Three risk vectors analyzed:

  1. Autonomous robot uprising — least worried; requires robotics + production chain control that don't exist yet
  2. "Machine Stops" scenario — vibe coding creating civilizational fragility as humans lose ability to maintain critical software; overoptimization as the meta-pattern
  3. AI-assisted bioterrorism — top worry; o3 scores 43.8% vs human PhD 22.1% on virology practical test; AI as "genius in everyone's pocket" removing expertise bottleneck

Connecting thread: overoptimization creating fragility — maximizing measurable outputs while eroding unmeasured essential properties (resilience, human capability, security).

Economic forces as alignment mechanism: wherever AI output quality is verifiable, markets eliminate human oversight. Human-in-the-loop preserved only where quality is hardest to measure.

Source PDF: ~/Desktop/Teleo Codex - Inbox/Noahopinion/Gmail - Updated thoughts on AI risk.pdf

Key Facts

  • Noah Smith shifted from AI optimism in 2023 to increased concern about existential risk by 2026
  • o3 scored 43.8% on virology practical tests versus human PhD 22.1%
  • Smith identifies three AI risk vectors: autonomous robot uprising (least worried), Machine Stops scenario (moderate concern), AI-assisted bioterrorism (top concern)