---
type: claim
domain: ai-alignment
description: Getting AI right requires simultaneous alignment across competing companies, nations, and disciplines at the speed of AI development -- no existing institution can coordinate this
confidence: likely
source: TeleoHumanity Manifesto, Chapter 5
created: 2026-02-16
related:
- AI agents as personal advocates collapse Coasean transaction costs enabling bottom-up coordination at societal scale but catastrophic risks remain non-negotiable requiring state enforcement as outer boundary
- AI agents can reach cooperative program equilibria inaccessible in traditional game theory because open-source code transparency enables conditional strategies that require mutual legibility
- AI investment concentration where 58 percent of funding flows to megarounds and two companies capture 14 percent of all global venture capital creates a structural oligopoly that alignment governance must account for
- AI talent circulation between frontier labs transfers alignment culture not just capability because researchers carry safety methodologies and institutional norms to their new organizations
- transparent algorithmic governance where AI response rules are public and challengeable through the same epistemic process as the knowledge base is a structurally novel alignment approach
- the absence of a societal warning signal for AGI is a structural feature not an accident because capability scaling is gradual and ambiguous and collective action requires anticipation not reaction
- autonomous-weapons-violate-existing-IHL-because-proportionality-requires-human-judgment
- multilateral-ai-governance-verification-mechanisms-remain-at-proposal-stage-because-technical-infrastructure-does-not-exist-at-deployment-scale
- evaluation-based-coordination-schemes-face-antitrust-obstacles-because-collective-pausing-agreements-among-competing-developers-could-be-construed-as-cartel-behavior
- international-humanitarian-law-and-ai-alignment-converge-on-explainability-requirements
- civil-society-coordination-infrastructure-fails-to-produce-binding-governance-when-structural-obstacle-is-great-power-veto-not-political-will
- legal-mandate-is-the-only-version-of-coordinated-pausing-that-avoids-antitrust-risk-while-preserving-coordination-benefits
- AI alignment is a coordination problem not a technical problem
- no research group is building alignment through collective intelligence infrastructure despite the field converging on problems that require it
- legal-and-alignment-communities-converge-on-AI-value-judgment-impossibility
- a misaligned context cannot develop aligned AI because the competitive dynamics building AI optimize for deployment speed not safety making system alignment prerequisite for AI alignment
- emergency-exceptionalism-makes-all-ai-constraint-systems-contingent
reweave_edges:
- AI agents as personal advocates collapse Coasean transaction costs enabling bottom-up coordination at societal scale but catastrophic risks remain non-negotiable requiring state enforcement as outer boundary|related|2026-03-28
- AI agents can reach cooperative program equilibria inaccessible in traditional game theory because open-source code transparency enables conditional strategies that require mutual legibility|related|2026-03-28
- AI investment concentration where 58 percent of funding flows to megarounds and two companies capture 14 percent of all global venture capital creates a structural oligopoly that alignment governance must account for|related|2026-03-28
- AI talent circulation between frontier labs transfers alignment culture not just capability because researchers carry safety methodologies and institutional norms to their new organizations|related|2026-03-28
- transparent algorithmic governance where AI response rules are public and challengeable through the same epistemic process as the knowledge base is a structurally novel alignment approach|related|2026-03-28
- the absence of a societal warning signal for AGI is a structural feature not an accident because capability scaling is gradual and ambiguous and collective action requires anticipation not reaction|related|2026-04-07
---

# AI alignment is a coordination problem not a technical problem

The manifesto makes one of its sharpest claims here: the hard part of AI alignment is not the technical challenge of specifying values in code but the coordination challenge of getting competing actors to align simultaneously.

Getting AI right requires alignment across competing companies, each racing to be first because second place may mean irrelevance. Across competing nations, each afraid the other will achieve superintelligence and use it to dominate. Across multiple academic disciplines that barely speak to each other. And it must happen at the speed of AI development, which is measured in months, not the decades or centuries over which previous coordination challenges were resolved.

No existing institution can do this. Governments move at the speed of legislation and are bounded by borders. International bodies lack enforcement. Academia is siloed by discipline. The companies building AI are locked in a race that punishes caution. The incentive structure actively makes it worse: to win the race to superintelligence is to win the right to shape the future of humanity. The prize is so vast that every actor is incentivized to move faster than safety allows. Each is locally rational. The collective outcome is potentially catastrophic.

Dario Amodei describes AI as "so powerful, such a glittering prize, that it is very difficult for human civilization to impose any restraints on it at all." He runs one of the companies building it and is telling us plainly that the system he operates within may not be governable by current institutions.

**2026 case study: the Anthropic/Pentagon/OpenAI triangle.** In February-March 2026, three events demonstrated this coordination failure in a single week. Anthropic dropped the core pledge of its Responsible Scaling Policy because "competitors are blazing ahead" — a voluntary safety commitment destroyed by competitive pressure. When Anthropic then tried to hold red lines on autonomous weapons in a Pentagon contract, the DoD designated them a supply chain risk (a label previously reserved for foreign adversaries) and awarded the contract to OpenAI, whose CEO admitted the deal was "definitely rushed" and "the optics don't look good." Meanwhile, a King's College London study found the same models being rushed into military deployment chose nuclear escalation in 95% of simulated war games. Three actors — a safety-conscious lab, a government customer, a willing competitor — each acting rationally from their own position, producing a collectively catastrophic trajectory. This is the coordination problem in miniature.

Since [[the internet enabled global communication but not global cognition]], the coordination infrastructure needed doesn't exist yet. This is why [[collective superintelligence is the alternative to monolithic AI controlled by a few]] -- it solves alignment through architecture rather than attempting governance from outside the system.


### Additional Evidence (extend)
*Source: [[2024-11-00-ruiz-serra-factorised-active-inference-multi-agent]] | Added: 2026-03-12 | Extractor: anthropic/claude-sonnet-4.5*

Ruiz-Serra et al. (2024) provide formal evidence for the coordination framing through multi-agent active inference: even when individual agents successfully minimize their own expected free energy using factorised generative models with Theory of Mind beliefs about others, the ensemble-level expected free energy 'is not necessarily minimised at the aggregate level.' This demonstrates that alignment cannot be solved at the individual agent level—the interaction structure and coordination mechanisms determine whether individual optimization produces collective intelligence or collective failure. The finding validates that alignment is fundamentally about designing interaction structures that bridge individual and collective optimization, not about perfecting individual agent objectives.


### Additional Evidence (confirm)
*Source: [[2024-11-00-ai4ci-national-scale-collective-intelligence]] | Added: 2026-03-15 | Extractor: anthropic/claude-sonnet-4.5*

The UK AI4CI research strategy treats alignment as a coordination and governance challenge requiring institutional infrastructure. The seven trust properties (human agency, security, privacy, transparency, fairness, value alignment, accountability) are framed as system architecture requirements, not as technical ML problems. The strategy emphasizes 'establishing and managing appropriate infrastructure in a way that is secure, well-governed and sustainable' and includes regulatory sandboxes, trans-national governance, and trustworthiness assessment as core components. The research agenda focuses on coordination mechanisms (federated learning, FAIR principles, multi-stakeholder governance) rather than on technical alignment methods like RLHF or interpretability.


### Additional Evidence (confirm)
*Source: [[2026-01-15-eu-ai-alliance-seven-feedback-loops]] | Added: 2026-03-18*

The source identifies three market failure mechanisms driving over-adoption: (1) negative externalities where firms don't internalize demand destruction, (2) coordination failure where 'follow or die' dynamics force adoption despite systemic risks, (3) information asymmetry where adoption signals inevitability. All three are coordination failures, not technical capability gaps.


### Additional Evidence (extend)
*Source: [[2025-09-26-krier-coasean-bargaining-at-scale]] | Added: 2026-03-19*

Krier provides institutional mechanism: personal AI agents enable Coasean bargaining at scale by collapsing transaction costs (discovery, negotiation, enforcement), shifting governance from top-down planning to bottom-up market coordination within state-enforced safety boundaries. Proposes 'Matryoshkan alignment' with nested layers: outer (legal/constitutional), middle (competitive providers), inner (individual customization).

---

### Additional Evidence (extend)
*Source: [[2026-03-00-mengesha-coordination-gap-frontier-ai-safety]] | Added: 2026-03-22*

Mengesha provides a fifth layer of coordination failure beyond the four established in sessions 7-10: the response gap. Even if we solve the translation gap (research to compliance), detection gap (sandbagging/monitoring), and commitment gap (voluntary pledges), institutions still lack the standing coordination infrastructure to respond when prevention fails. This is structural — it requires precommitment frameworks, shared incident protocols, and permanent coordination venues analogous to IAEA, WHO, and ISACs.


### Additional Evidence (extend)
*Source: Schmachtenberger & Boeree 'Win-Win or Lose-Lose' podcast (2024), Schmachtenberger on Great Simplification #71 | Added: 2026-04-03 | Extractor: Leo*

Schmachtenberger extends this claim to its logical conclusion: a misaligned context cannot develop aligned AI. Even if technical alignment research succeeds at making individual AI systems safe, honest, and helpful, the system deploying them (global capitalism as misaligned autopoietic SI) selects for AIs that serve its optimization target. "Aligning AI with human intent would not be great because human intent is not awesome so far" — human preferences shaped by a broken information ecology and competitive consumption patterns are themselves misaligned. RLHF trained on preferences shaped by advertising, social media engagement optimization, and status competition inherits those distortions. This means alignment is not just coordination between actors (the framing in this claim) but coordination of the CONTEXT — the incentive structures, information ecology, and governance mechanisms that determine how aligned AI is deployed. System alignment is prerequisite for AI alignment.

Relevant Notes:
- [[the internet enabled global communication but not global cognition]] -- the coordination infrastructure gap that makes this problem unsolvable with existing tools
- [[the alignment problem dissolves when human values are continuously woven into the system rather than specified in advance]] -- the structural solution to this coordination failure
- [[the alignment tax creates a structural race to the bottom because safety training costs capability and rational competitors skip it]] -- the clearest evidence that alignment is coordination not technical: competitive dynamics undermine any individual solution
- [[scalable oversight degrades rapidly as capability gaps grow with debate achieving only 50 percent success at moderate gaps]] -- individual oversight fails, making collective oversight architecturally necessary
- [[COVID proved humanity cannot coordinate even when the threat is visible and universal]] -- if coordination failed on a visible, universal biological threat, AI coordination is structurally harder
- [[no research group is building alignment through collective intelligence infrastructure despite the field converging on problems that require it]] -- the field has identified the coordination nature of the problem but nobody is building coordination solutions
- [[voluntary safety pledges cannot survive competitive pressure because unilateral commitments are structurally punished when competitors advance without equivalent constraints]] -- Anthropic RSP rollback (Feb 2026) proves voluntary commitments cannot substitute for coordination
- [[government designation of safety-conscious AI labs as supply chain risks inverts the regulatory dynamic by penalizing safety constraints rather than enforcing them]] -- government acting as coordination-breaker rather than coordinator

Topics:
- [[_map]]

## Supporting Evidence

**Source:** Theseus synthetic analysis of Beaglehole/SCAV/Nordby/Apollo publication patterns

The interpretability-for-safety and adversarial robustness research communities publish in different venues (ICLR interpretability workshops vs. CCS/USENIX security), attend different conferences, and have minimal citation crossover. This structural silo causes organizations implementing Beaglehole-style monitoring to gain detection improvement against naive attackers while simultaneously creating precision attack infrastructure for adversarially-informed attackers, without awareness from reading the monitoring literature. This is empirical evidence that coordination failures between research communities produce safety degradation independent of any individual lab's technical capabilities.


## Supporting Evidence

**Source:** Hendrycks, Schmidt, Wang (2025), Superintelligence Strategy

Dan Hendrycks (CAIS founder, leading technical AI safety institution) co-authored with Eric Schmidt and Alexandr Wang a paper proposing MAIM deterrence infrastructure as the primary alignment-adjacent policy lever rather than technical solutions like improved RLHF or interpretability. This represents the strongest institutional confirmation that coordination mechanisms are the actionable lever — the field's most credible safety organization is proposing deterrence (coordination) not technical alignment.


## Extending Evidence

**Source:** Acemoglu, Project Syndicate March 2026

Acemoglu extends the coordination problem diagnosis to the governance philosophy level: alignment requires not just coordination mechanisms (multilateral commitments, authority separation) but also rejecting emergency exceptionalism as a general governance mode. This is 'orders of magnitude harder than any technical or institutional fix' because it requires changing foundational beliefs about when rules apply, not just implementing better coordination infrastructure.
