bc6ab0d368
Pipeline auto-fixer: removed [[ ]] brackets from links that don't resolve to existing claims in the knowledge base.
38 lines
3 KiB
Markdown
38 lines
3 KiB
Markdown
---
|
|
type: claim
|
|
domain: ai-alignment
|
|
description: "Formal impossibility result: no RLHF system can simultaneously achieve epsilon-representativeness, polynomial tractability, and delta-robustness"
|
|
confidence: likely
|
|
source: "Sahoo et al. (Berkeley AI Safety Initiative, AWS, Meta, Stanford, Northeastern), NeurIPS 2025 Workshop on Socially Responsible and Trustworthy Foundation Models"
|
|
created: 2026-03-11
|
|
depends_on: ["RLHF and DPO both fail at preference diversity because they assume a single reward function can capture context-dependent human values"]
|
|
---
|
|
|
|
# RLHF alignment trilemma proves no system can simultaneously achieve representativeness, tractability, and robustness
|
|
|
|
The alignment trilemma establishes a formal impossibility result: no RLHF system can simultaneously achieve three critical properties:
|
|
|
|
1. **Epsilon-representativeness** across diverse human values
|
|
2. **Polynomial tractability** in sample and compute complexity
|
|
3. **Delta-robustness** against adversarial perturbations and distribution shift
|
|
|
|
This is proven through complexity theory, not an implementation limitation. The core complexity bound shows that achieving both representativeness (epsilon ≤ 0.01) and robustness (delta ≤ 0.001) for global-scale populations requires Ω(2^{d_context}) operations—super-polynomial in context dimensionality. This makes the combination computationally intractable for real-world deployment.
|
|
|
|
The paper identifies three strategic relaxation pathways, each abandoning one vertex of the trilemma:
|
|
|
|
1. **Constrain representativeness**: Focus on K << |H| "core" human values (~30 universal principles) rather than attempting global diversity
|
|
2. **Scope robustness narrowly**: Define restricted adversarial classes targeting only plausible threats rather than worst-case perturbations
|
|
3. **Accept super-polynomial costs**: Justify exponential compute for high-stakes applications where tractability can be relaxed
|
|
|
|
Critically, this result arrives at a compatible impossibility conclusion to [[universal alignment is mathematically impossible because Arrows impossibility theorem applies to aggregating diverse human preferences into a single coherent objective]] but through an independent mathematical tradition (complexity theory rather than social choice theory). This provides convergent evidence from different intellectual foundations that universal alignment faces fundamental mathematical barriers.
|
|
|
|
---
|
|
|
|
Relevant Notes:
|
|
- [[RLHF and DPO both fail at preference diversity because they assume a single reward function can capture context-dependent human values]]
|
|
- [[universal alignment is mathematically impossible because Arrows impossibility theorem applies to aggregating diverse human preferences into a single coherent objective]]
|
|
- [[safe AI development requires building alignment mechanisms before scaling capability]]
|
|
- [[pluralistic alignment must accommodate irreducibly diverse values simultaneously rather than converging on a single aligned state]]
|
|
|
|
Topics:
|
|
- domains/ai-alignment/_map
|