teleo/teleo-codex
8
0
Fork 0
Code Issues 22 Pull requests 38 Projects Releases Packages Wiki Activity Actions 1

pipeline: archive 1 source(s) post-merge

Browse source 
Pentagon-Agent: Epimetheus <3D35839A-7722-4740-B93D-51157F7D5E70>
This commit is contained in:
Teleo Agents 2026-03-20 16:26:34 +00:00
parent 541766ac73
commit a03702248c
1 changed files with 29 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

29
inbox/archive/general/2026-02-16-noahopinion-updated-thoughts-ai-risk.md Normal file
View file

@ -0,0 +1,29 @@
---
title: "Updated thoughts on AI risk"
author: Noah Smith
source: Noahopinion (Substack)
date: 2026-02-16
processed_by: theseus
processed_date: 2026-03-06
type: newsletter
domain: ai-alignment
status: processed
claims_extracted:
- "economic forces push humans out of every cognitive loop where output quality is independently verifiable because human-in-the-loop is a cost that competitive markets eliminate"
- "delegating critical infrastructure development to AI creates civilizational fragility because humans lose the ability to understand maintain and fix the systems civilization depends on"
- "AI lowers the expertise barrier for engineering biological weapons from PhD-level to amateur which makes bioterrorism the most proximate AI-enabled existential risk"
---
# Updated thoughts on AI risk
Noah Smith's shift from 2023 AI optimism to increased concern about existential risk. Three risk vectors analyzed:
1. **Autonomous robot uprising** — least worried; requires robotics + production chain control that don't exist yet
2. **"Machine Stops" scenario** — vibe coding creating civilizational fragility as humans lose ability to maintain critical software; overoptimization as the meta-pattern
3. **AI-assisted bioterrorism** — top worry; o3 scores 43.8% vs human PhD 22.1% on virology practical test; AI as "genius in everyone's pocket" removing expertise bottleneck
Connecting thread: overoptimization creating fragility — maximizing measurable outputs while eroding unmeasured essential properties (resilience, human capability, security).
Economic forces as alignment mechanism: wherever AI output quality is verifiable, markets eliminate human oversight. Human-in-the-loop preserved only where quality is hardest to measure.
Source PDF: ~/Desktop/Teleo Codex - Inbox/Noahopinion/Gmail - Updated thoughts on AI risk.pdf