36 lines
2.2 KiB
Markdown
36 lines
2.2 KiB
Markdown
---
|
|
type: source
|
|
title: "Distributional AGI Safety"
|
|
author: "Nenad Tomašev, Matija Franklin, Julian Jacobs, Sébastien Krier, Simon Osindero"
|
|
url: https://arxiv.org/abs/2512.16856
|
|
date_published: 2025-12-18
|
|
date_archived: 2026-03-16
|
|
domain: ai-alignment
|
|
status: enrichment
|
|
processed_by: theseus
|
|
tags: [distributed-agi, multi-agent-safety, patchwork-hypothesis, coordination]
|
|
sourced_via: "Alex Obadia (@ObadiaAlex) tweet, ARIA Research Scaling Trust programme"
|
|
twitter_id: "712705562191011841"
|
|
processed_by: theseus
|
|
processed_date: 2026-03-19
|
|
enrichments_applied: ["AGI may emerge as a patchwork of coordinating sub-AGI agents rather than a single monolithic system.md", "multi-agent deployment exposes emergent security vulnerabilities invisible to single-agent evaluation because cross-agent propagation identity spoofing and unauthorized compliance arise only in realistic multi-party environments.md", "AI alignment is a coordination problem not a technical problem.md"]
|
|
extraction_model: "anthropic/claude-sonnet-4.5"
|
|
---
|
|
|
|
# Distributional AGI Safety
|
|
|
|
Tomašev et al. challenge the monolithic AGI assumption. They propose the "patchwork AGI hypothesis" — general capability levels first manifest through coordination among groups of sub-AGI agents with complementary skills and affordances, not through a single unified system.
|
|
|
|
Key arguments:
|
|
- AI safety research has focused on safeguarding individual systems, overlooking distributed emergence
|
|
- Rapid deployment of agents with tool-use and coordination capabilities makes distributed safety urgent
|
|
- Proposed framework: "virtual agentic sandbox economies" with robust market mechanisms, auditability, reputation management, and oversight for collective risks
|
|
- Safety focus shifts from individual agent alignment to managing risks at the system-of-systems level
|
|
|
|
Directly relevant to our claim [[AGI may emerge as a patchwork of coordinating sub-AGI agents rather than a single monolithic system]] and to the collective superintelligence thesis.
|
|
|
|
|
|
## Key Facts
|
|
- ARIA Research Scaling Trust programme funded research on distributional AGI safety
|
|
- Paper published December 2025 on arXiv (2512.16856)
|
|
- Authors: Nenad Tomašev, Matija Franklin, Julian Jacobs, Sébastien Krier, Simon Osindero
|