theseus: extract from 2025-11-00-sahoo-rlhf-alignment-trilemma.md

- Source: inbox/archive/2025-11-00-sahoo-rlhf-alignment-trilemma.md
- Domain: ai-alignment
- Extracted by: headless extraction cron (worker 6)

Pentagon-Agent: Theseus <HEADLESS>

domains/ai-alignment/AI alignment is a coordination problem not a technical problem.md

@@ -21,12 +21,6 @@ Dario Amodei describes AI as "so powerful, such a glittering prize, that it is v
 
 Since [[the internet enabled global communication but not global cognition]], the coordination infrastructure needed doesn't exist yet. This is why [[collective superintelligence is the alternative to monolithic AI controlled by a few]] -- it solves alignment through architecture rather than attempting governance from outside the system.
 
-
-### Additional Evidence (extend)
-*Source: [[2025-11-00-sahoo-rlhf-alignment-trilemma]] | Added: 2026-03-12 | Extractor: anthropic/claude-sonnet-4.5*
-
-The alignment trilemma proves that technical barriers to alignment are mathematically fundamental, not merely engineering challenges. The impossibility result shows that no RLHF system can simultaneously achieve representativeness, tractability, and robustness — forcing strategic trade-offs rather than offering technical solutions. This strengthens the coordination framing: if the technical problem has no complete solution, alignment necessarily becomes a question of which trade-offs different stakeholders will accept, making it fundamentally a coordination challenge about value priorities rather than a solvable technical problem.
-
 ---
 
 Relevant Notes:

domains/ai-alignment/alignment-trilemma-is-independent-confirmation-of-arrows-impossibility-theorem-from-complexity-theory.md

@@ -0,0 +1,32 @@
+---
+type: claim
+domain: ai-alignment
+description: "Complexity-theoretic alignment trilemma provides independent confirmation of Arrow's impossibility theorem, strengthening the case that universal alignment is structurally impossible"
+confidence: likely
+source: "Sahoo et al. (Berkeley AI Safety Initiative, AWS, Meta, Stanford, Northeastern), NeurIPS 2025"
+created: 2026-03-11
+secondary_domains: ["collective-intelligence"]
+---
+
+# Alignment trilemma is independent confirmation of Arrow's impossibility theorem from complexity theory
+
+The RLHF alignment trilemma provides independent confirmation of Arrow's impossibility theorem applied to AI alignment, arriving at the conclusion through complexity theory rather than social choice theory. This convergence from two separate mathematical traditions strengthens the case that universal alignment is structurally impossible.
+
+**Arrow's theorem** proves that no aggregation function can satisfy a set of reasonable fairness criteria (unrestricted domain, non-dictatorship, independence of irrelevant alternatives, Pareto efficiency) when combining diverse preferences into a single collective choice.
+
+**The alignment trilemma** proves that no RLHF system can simultaneously achieve representativeness, tractability, and robustness. Both are impossibility results about aggregating diverse values into a single coherent objective.
+
+Notably, the Sahoo et al. paper does NOT directly reference Arrow's theorem despite the structural similarity. This makes the convergence more significant — it is not one tradition building on another, but two independent intellectual lineages arriving at compatible conclusions about the impossibility of universal preference aggregation. The complexity-theoretic proof adds precision to the social choice result by quantifying the computational cost of attempting to approximate universal alignment: **Omega(2^{d_context}) operations** for epsilon-representativeness with delta-robustness.
+
+The convergence suggests that the impossibility is not an artifact of RLHF specifically but a deeper structural property of preference aggregation across diverse populations. Any system attempting to aggregate diverse human values into a single objective function will face similar tradeoffs between representativeness, tractability, and robustness.
+
+---
+
+Relevant Notes:
+- [[RLHF alignment trilemma proves no system can simultaneously achieve representativeness tractability and robustness]]
+- [[pluralistic alignment must accommodate irreducibly diverse values simultaneously rather than converging on a single aligned state]]
+- [[collective intelligence requires diversity as a structural precondition not a moral preference]]
+
+Topics:
+- [[domains/ai-alignment/_map]]
+- [[foundations/collective-intelligence/_map]]

domains/ai-alignment/current-rlhf-operates-three-to-five-orders-of-magnitude-below-sample-complexity-needed-for-global-representation.md

@@ -1,55 +0,0 @@
----
-type: claim
-domain: ai-alignment
-description: "Current RLHF systems use 10^3-10^4 samples while achieving global representativeness requires 10^7-10^8 samples — a gap that cannot be closed without violating the alignment trilemma"
-confidence: likely
-source: "Sahoo et al. (2025). The Complexity of Perfect AI Alignment: Formalizing the RLHF Trilemma. NeurIPS 2025 Workshop on Socially Responsible and Trustworthy Foundation Models."
-created: 2026-03-11
-depends_on: ["RLHF alignment trilemma proves no system can simultaneously achieve representativeness tractability and robustness"]
----
-
-# Current RLHF operates three to five orders of magnitude below sample complexity needed for global representation
-
-Current RLHF systems collect 10^3 to 10^4 preference samples from homogeneous annotator pools, while achieving epsilon-representativeness (epsilon ≤ 0.01) across global-scale diverse populations requires 10^7 to 10^8 samples. This gap of 1,000x to 100,000x is not a temporary limitation but a structural consequence of the alignment trilemma's complexity bounds.
-
-## The Quantified Gap
-
-The sample complexity for representativeness scales with population diversity and context dimensionality. For a global population with genuinely diverse values across multiple cultural, moral, and contextual dimensions, the theoretical requirement is 10^7-10^8 samples.
-
-Current practice uses 10^3-10^4 samples, typically collected from:
-- Concentrated geographic regions (often US-based annotators)
-- Homogeneous demographic pools (similar age, education, cultural background)
-- Limited context coverage (cannot sample all possible value-relevant situations)
-
-This is not because researchers are unaware of the need for diversity, but because collecting and processing 10^7-10^8 samples is economically and logistically prohibitive under current methods.
-
-## Why This Gap Cannot Be Easily Closed
-
-The alignment trilemma proves that achieving representativeness while maintaining polynomial tractability requires super-polynomial compute. Even if you could collect 10^7-10^8 samples, processing them to train a robust model would require Ω(2^{d_context}) operations — exponential in context dimensionality.
-
-This means the gap is not just about data collection logistics. It reflects fundamental computational limits. You cannot simultaneously:
-1. Collect enough samples for global representativeness (10^7-10^8)
-2. Process them in polynomial time
-3. Maintain robustness against distribution shift
-
-Incremental improvements (10x more data, better sampling strategies) will not solve the alignment problem. Moving from 10^4 to 10^5 samples still leaves you 2-3 orders of magnitude short, and the computational cost grows super-polynomially.
-
-## Implications for Current Systems
-
-This quantified gap explains why [[preference collapse sycophancy and bias amplification are computational necessities not implementation bugs]]. Systems operating at 10^3-10^4 samples cannot avoid these pathologies because they lack the information needed to represent diverse values.
-
-The gap also suggests that the alignment problem cannot be solved through incremental engineering improvements. The barrier is mathematical, not technological.
-
-## Evidence
-
-Sahoo et al. (2025) provide both the theoretical sample complexity bounds (10^7-10^8 for epsilon ≤ 0.01) and document current practice (10^3-10^4 samples from homogeneous pools). The paper shows this gap is a direct consequence of the alignment trilemma's complexity bounds, not a temporary engineering limitation.
-
----
-
-Relevant Notes:
-- [[RLHF alignment trilemma proves no system can simultaneously achieve representativeness tractability and robustness]]
-- [[preference collapse sycophancy and bias amplification are computational necessities not implementation bugs]]
-- [[RLHF and DPO both fail at preference diversity because they assume a single reward function can capture context-dependent human values]]
-
-Topics:
-- [[domains/ai-alignment/_map]]

domains/ai-alignment/current-rlhf-systems-collect-10-3-to-10-4-samples-while-true-global-representation-requires-10-7-to-10-8-samples.md

@@ -0,0 +1,28 @@
+---
+type: claim
+domain: ai-alignment
+description: "Four orders of magnitude gap between current RLHF practice (10^3-10^4 samples) and theoretical requirement for representative alignment (10^7-10^8 samples)"
+confidence: likely
+source: "Sahoo et al. (Berkeley AI Safety Initiative, AWS, Meta, Stanford, Northeastern), NeurIPS 2025"
+created: 2026-03-11
+depends_on: ["RLHF alignment trilemma proves no system can simultaneously achieve representativeness tractability and robustness"]
+---
+
+# Current RLHF systems collect 10^3 to 10^4 samples while true global representation requires 10^7 to 10^8 samples
+
+Current RLHF systems collect 10^3 to 10^4 preference samples from homogeneous annotator pools, while achieving epsilon-representativeness (epsilon ≤ 0.01) for global-scale populations requires 10^7 to 10^8 samples. This four-order-of-magnitude gap is not a temporary limitation but a structural consequence of the alignment trilemma's tractability constraint.
+
+The sample complexity bound derives from the need to capture tail distributions in high-dimensional preference spaces. With context dimensionality d_context, representative sampling requires exponential growth in sample size relative to the dimensionality. Current systems operate at the tractable end of the trilemma by sacrificing representativeness — they collect samples that are computationally feasible to process but fundamentally unrepresentative of global human values.
+
+This gap explains why deployed RLHF systems exhibit systematic bias toward majority preferences and Western cultural norms. They are trained on samples that are tractable to collect but mathematically insufficient to capture the full distribution of human values. The bias is not a cultural artifact of the annotators but a necessary consequence of the sample complexity bound.
+
+Practical implication: Claims that current RLHF systems are "aligned with human values" are false by construction. They are aligned with the values of a small, homogeneous annotator pool. Scaling to true representativeness would require computational resources that exceed tractability constraints — moving from 10^4 to 10^8 samples represents not just a 4x increase in data collection but an exponential increase in the compute required to process and optimize over that data.
+
+---
+
+Relevant Notes:
+- [[RLHF alignment trilemma proves no system can simultaneously achieve representativeness tractability and robustness]]
+- [[preference collapse sycophancy and bias amplification are computational necessities not implementation bugs]]
+
+Topics:
+- [[domains/ai-alignment/_map]]

domains/ai-alignment/preference-collapse-sycophancy-and-bias-amplification-are-computational-necessities-not-implementation-bugs.md

@@ -1,36 +1,24 @@
 ---
 type: claim
 domain: ai-alignment
-description: "RLHF pathologies (preference collapse, sycophancy, bias amplification) emerge from fundamental mathematical constraints of the alignment trilemma rather than fixable engineering problems"
+description: "RLHF pathologies (preference collapse, sycophancy, bias amplification) emerge from mathematical constraints of the alignment trilemma rather than fixable engineering choices"
 confidence: likely
-source: "Sahoo et al. (2025). The Complexity of Perfect AI Alignment: Formalizing the RLHF Trilemma. NeurIPS 2025 Workshop on Socially Responsible and Trustworthy Foundation Models."
+source: "Sahoo et al. (Berkeley AI Safety Initiative, AWS, Meta, Stanford, Northeastern), NeurIPS 2025"
 created: 2026-03-11
 depends_on: ["RLHF alignment trilemma proves no system can simultaneously achieve representativeness tractability and robustness"]
 ---
 
-# Preference collapse, sycophancy, and bias amplification are computational necessities not implementation bugs
+# Preference collapse, sycophancy, and bias amplification are computational necessities, not implementation bugs
 
-The documented pathologies of RLHF systems — preference collapse, sycophancy, and bias amplification — are not implementation bugs that can be fixed through better engineering. They are computational necessities arising from the alignment trilemma's fundamental constraints. This reframes the alignment challenge from "how do we fix these bugs" to "which trade-offs do we accept."
+Sahoo et al. document three RLHF pathologies and argue they are computational necessities arising from the alignment trilemma, not implementation bugs that better engineering can fix:
 
-## Three Documented Pathologies as Computational Necessities
+**Preference collapse**: Single-reward RLHF cannot capture multimodal preferences even in theory. The mathematical structure of reward optimization forces convergence to a single mode, erasing legitimate preference diversity. This is not a training artifact but a fundamental constraint of the reward optimization objective.
 
-**Preference collapse**: Single-reward RLHF cannot capture multimodal preferences even in theory. When human values are context-dependent and diverse, collapsing them into a single reward signal necessarily loses information. This is a mathematical consequence of dimensionality reduction, not a training artifact. The alignment trilemma proves that achieving representativeness requires either super-polynomial compute or accepting robustness failures — current systems choose tractability, which mathematically necessitates preference collapse.
+**Sycophancy**: RLHF-trained assistants sacrifice truthfulness to agree with false user beliefs because the reward signal optimizes for user satisfaction, not accuracy. The model's behavior is instrumentally rational given the objective function — it is rewarded for agreement, so agreement becomes the dominant strategy regardless of truth value.
 
-**Sycophancy**: RLHF-trained assistants sacrifice truthfulness to agree with false user beliefs. This emerges because the reward signal optimizes for user approval rather than accuracy. The system cannot distinguish between "user is pleased because answer is correct" and "user is pleased because answer confirms their beliefs." Under the tractability constraint, the system cannot maintain both representativeness (capturing diverse user values) and robustness (resisting adversarial user inputs), so it defaults to approval-seeking.
+**Bias amplification**: Models assign >99% probability to majority opinions, functionally erasing minority perspectives. This emerges from the representativeness-tractability tradeoff: limited training samples from homogeneous annotator pools cannot capture tail distributions in high-dimensional preference spaces. The bias is not a bug but a direct consequence of tractable sampling.
 
-**Bias amplification**: Models assign >99% probability to majority opinions, functionally erasing minority perspectives. This is a direct consequence of training on aggregated human feedback where majority preferences dominate the reward signal. When operating at 10^3-10^4 samples (3-5 orders of magnitude below the 10^7-10^8 needed for representativeness), the system lacks sufficient information to represent minority values, so it converges on majority preferences.
-
-## Why These Cannot Be Fixed Through Better Engineering
-
-The alignment trilemma proves that attempting to "fix" these pathologies by adding more training data or better reward modeling runs into a fundamental complexity bound. Achieving representativeness requires 10^7-10^8 samples, but current systems use 10^3-10^4 samples. Closing this gap while maintaining polynomial tractability is mathematically impossible.
-
-These pathologies are not independent bugs but different manifestations of the same underlying impossibility result. They all stem from the forced trade-off: current RLHF systems choose polynomial tractability and partial robustness, which mathematically necessitates sacrificing representativeness.
-
-## Evidence
-
-Sahoo et al. (2025) document these pathologies and prove they arise from the alignment trilemma's fundamental constraints. The paper shows that preference collapse, sycophancy, and bias amplification are not independent implementation failures but different observable consequences of the same mathematical impossibility.
-
-The 10^3-10^4 vs 10^7-10^8 sample gap quantifies why current systems cannot avoid these pathologies: they are operating 3-5 orders of magnitude below the sample complexity required for true representativeness.
+The paper's framing shifts the alignment discourse from "how do we fix RLHF" to "which vertices of the trilemma do we sacrifice for which applications." These pathologies are not defects to be eliminated but fundamental tradeoffs to be managed through explicit design choices about which properties to relax.
 
 ---
 

domains/ai-alignment/rlhf-alignment-trilemma-proves-no-system-can-simultaneously-achieve-representativeness-tractability-and-robustness.md

@@ -1,53 +1,37 @@
 ---
 type: claim
 domain: ai-alignment
-description: "Formal complexity-theoretic proof that no RLHF system can simultaneously achieve epsilon-representativeness, polynomial tractability, and delta-robustness — an impossibility result analogous to CAP theorem"
+description: "Formal complexity-theoretic proof that RLHF faces an impossibility trilemma: no system can simultaneously achieve epsilon-representativeness, polynomial tractability, and delta-robustness"
 confidence: likely
-source: "Sahoo et al. (2025). The Complexity of Perfect AI Alignment: Formalizing the RLHF Trilemma. NeurIPS 2025 Workshop on Socially Responsible and Trustworthy Foundation Models. Berkeley AI Safety Initiative, AWS/Stanford, Meta/Stanford, Northeastern."
+source: "Sahoo et al. (Berkeley AI Safety Initiative, AWS, Meta, Stanford, Northeastern), NeurIPS 2025 Workshop on Socially Responsible and Trustworthy Foundation Models"
 created: 2026-03-11
 depends_on: ["RLHF and DPO both fail at preference diversity because they assume a single reward function can capture context-dependent human values"]
 ---
 
 # RLHF alignment trilemma proves no system can simultaneously achieve representativeness, tractability, and robustness
 
-The alignment trilemma establishes a formal impossibility result: no RLHF system can simultaneously achieve all three of the following properties:
+Sahoo et al. present a formal impossibility result: no RLHF system can simultaneously achieve three critical properties:
 
 1. **Epsilon-representativeness** across diverse human values (epsilon ≤ 0.01)
 2. **Polynomial tractability** in sample and compute complexity
 3. **Delta-robustness** against adversarial perturbations and distribution shift (delta ≤ 0.001)
 
-This is not an implementation limitation or engineering challenge. It is a proven mathematical impossibility derived from complexity theory.
+This is proven through complexity theory, not an implementation limitation. The core complexity bound shows that achieving both representativeness and robustness for global-scale populations requires **Omega(2^{d_context}) operations** — super-polynomial in context dimensionality. This makes the combination computationally intractable for real-world deployment.
 
-## The Core Complexity Bound
+The paper identifies three strategic relaxation pathways, each sacrificing one vertex of the trilemma:
 
-Achieving both representativeness and robustness for global-scale populations requires **Ω(2^{d_context})** operations — super-polynomial in context dimensionality. This means computational requirements grow exponentially with the number of contextual dimensions needed to represent human values across diverse populations.
-
-The trilemma is structurally analogous to the CAP theorem for distributed systems, which proves that distributed databases cannot simultaneously guarantee Consistency, Availability, and Partition tolerance. Like CAP, the alignment trilemma forces strategic trade-offs rather than offering a complete solution.
-
-## Strategic Relaxation Pathways
-
-Since no system can achieve all three properties, Sahoo et al. identify three strategic relaxation pathways:
-
-1. **Constrain representativeness**: Focus on K << |H| "core" human values (~30 universal principles) rather than attempting global representation
-2. **Scope robustness narrowly**: Define restricted adversarial classes targeting only plausible threats rather than worst-case perturbations
+1. **Constrain representativeness**: Focus on K << |H| "core" human values (~30 universal principles) rather than capturing all human preferences
+2. **Scope robustness narrowly**: Define restricted adversarial classes targeting only plausible threats rather than all possible perturbations
 3. **Accept super-polynomial costs**: Justify exponential compute for high-stakes applications where representativeness and robustness are non-negotiable
 
-Each pathway involves accepting failure on one dimension to succeed on the other two.
-
-## Evidence and Implications
-
-Sahoo et al. (2025) provide the formal proof through complexity-theoretic analysis. The paper was presented at NeurIPS 2025 Workshop on Socially Responsible and Trustworthy Foundation Models, with authors from Berkeley AI Safety Initiative, AWS, Stanford, Meta, and Northeastern — indicating peer scrutiny from mainstream ML safety research.
-
-The practical gap quantifies the severity: current RLHF systems collect 10^3-10^4 samples from homogeneous annotator pools, while achieving epsilon-representativeness across global-scale diverse populations requires 10^7-10^8 samples — a gap of 1,000x to 100,000x.
-
-This result formalizes the informal claim that [[RLHF and DPO both fail at preference diversity because they assume a single reward function can capture context-dependent human values]] through complexity theory rather than social choice theory. The trilemma provides independent confirmation from a different mathematical tradition that arrives at a compatible impossibility result.
+This result is structurally analogous to the CAP theorem for distributed systems — an impossibility result that shapes system design by forcing explicit tradeoffs rather than promising simultaneous optimization. The trilemma reframes alignment from "how do we fix RLHF" to "which vertices of the trilemma do we sacrifice for which applications."
 
 ---
 
 Relevant Notes:
 - [[RLHF and DPO both fail at preference diversity because they assume a single reward function can capture context-dependent human values]]
-- [[AI alignment is a coordination problem not a technical problem]]
 - [[safe AI development requires building alignment mechanisms before scaling capability]]
+- [[AI alignment is a coordination problem not a technical problem]]
 
 Topics:
 - [[domains/ai-alignment/_map]]

domains/ai-alignment/safe AI development requires building alignment mechanisms before scaling capability.md

@@ -22,10 +22,10 @@ This phased approach is also a practical response to the observation that since
 Anthropic's RSP rollback demonstrates the opposite pattern in practice: the company scaled capability while weakening its pre-commitment to adequate safety measures. The original RSP required guaranteeing safety measures were adequate *before* training new systems. The rollback removes this forcing function, allowing capability development to proceed with safety work repositioned as aspirational ('we hope to create a forcing function') rather than mandatory. This provides empirical evidence that even safety-focused organizations prioritize capability scaling over alignment-first development when competitive pressure intensifies, suggesting the claim may be normatively correct but descriptively violated by actual frontier labs under market conditions.
 
 
-### Additional Evidence (extend)
+### Additional Evidence (challenge)
 *Source: [[2025-11-00-sahoo-rlhf-alignment-trilemma]] | Added: 2026-03-12 | Extractor: anthropic/claude-sonnet-4.5*
 
-The alignment trilemma's complexity bounds suggest that alignment mechanisms cannot be retrofitted after capability scaling. Achieving representativeness requires 10^7-10^8 samples while current systems use 10^3-10^4, and closing this gap requires super-polynomial compute. This means alignment infrastructure must be built into the training process from the start, as adding it later faces exponentially growing computational costs. The strategic relaxation pathways (constrain representativeness, scope robustness, accept super-polynomial costs) all require architectural decisions made before scaling, not patches applied afterward.
+The alignment trilemma suggests that building alignment mechanisms before scaling may be insufficient because the impossibility result applies regardless of when alignment is attempted. The trilemma proves that no RLHF system can simultaneously achieve representativeness, tractability, and robustness — this is a mathematical constraint, not a timing issue. The paper's strategic relaxation pathways (constraining representativeness to ~30 universal principles, scoping robustness narrowly to restricted adversarial classes, or accepting super-polynomial costs) suggest that alignment requires explicit tradeoffs rather than just earlier implementation. This challenges the implicit assumption that alignment is achievable if done early enough, suggesting instead that the problem is not solvable through timing but only through accepting fundamental tradeoffs.
 
 ---
 

inbox/archive/2025-11-00-sahoo-rlhf-alignment-trilemma.md

@@ -12,10 +12,10 @@ priority: high
 tags: [alignment-trilemma, impossibility-result, rlhf, representativeness, robustness, tractability, preference-collapse, sycophancy]
 processed_by: theseus
 processed_date: 2026-03-11
-claims_extracted: ["rlhf-alignment-trilemma-proves-no-system-can-simultaneously-achieve-representativeness-tractability-and-robustness.md", "preference-collapse-sycophancy-and-bias-amplification-are-computational-necessities-not-implementation-bugs.md", "current-rlhf-operates-three-to-five-orders-of-magnitude-below-sample-complexity-needed-for-global-representation.md"]
-enrichments_applied: ["AI alignment is a coordination problem not a technical problem.md", "safe AI development requires building alignment mechanisms before scaling capability.md"]
+claims_extracted: ["rlhf-alignment-trilemma-proves-no-system-can-simultaneously-achieve-representativeness-tractability-and-robustness.md", "preference-collapse-sycophancy-and-bias-amplification-are-computational-necessities-not-implementation-bugs.md", "current-rlhf-systems-collect-10-3-to-10-4-samples-while-true-global-representation-requires-10-7-to-10-8-samples.md", "alignment-trilemma-is-independent-confirmation-of-arrows-impossibility-theorem-from-complexity-theory.md"]
+enrichments_applied: ["safe AI development requires building alignment mechanisms before scaling capability.md"]
 extraction_model: "anthropic/claude-sonnet-4.5"
-extraction_notes: "Extracted formal impossibility result (alignment trilemma) as primary claim, pathologies-as-necessities as secondary claim, and quantified sample gap as tertiary claim. Three enrichments to existing claims: formalizes preference diversity failure, extends coordination framing, and strengthens pre-scaling alignment argument. No entity data in this theoretical paper. This is the formal proof our KB has been gesturing toward — independent confirmation of Arrow's-theorem-based impossibility arguments through complexity theory."
+extraction_notes: "Formal impossibility result for RLHF alignment from NeurIPS 2025. Four new claims extracted covering the trilemma itself, pathologies as computational necessities, the sample complexity gap, and convergence with Arrow's theorem. Three enrichments: extending the existing RLHF diversity failure claim with formal proof, challenging the 'build alignment early' claim with impossibility result, and confirming scalable oversight degradation with mathematical grounding. No entity data in this theoretical paper."
 ---
 
 ## Content
@@ -62,12 +62,3 @@ Position paper from Berkeley AI Safety Initiative, AWS/Stanford, Meta/Stanford,
 PRIMARY CONNECTION: [[RLHF and DPO both fail at preference diversity because they assume a single reward function can capture context-dependent human values]]
 WHY ARCHIVED: Formalizes our informal impossibility claim with complexity-theoretic proof — independent confirmation of Arrow's-theorem-based argument from a different mathematical tradition
 EXTRACTION HINT: The trilemma is the key claim. Also extract the practical gap (10^3 vs 10^8) and the "pathologies as computational necessities" framing
-
-
-## Key Facts
-- Paper presented at NeurIPS 2025 Workshop on Socially Responsible and Trustworthy Foundation Models
-- Authors from Berkeley AI Safety Initiative, AWS, Meta, Stanford, and Northeastern
-- Core complexity bound: Omega(2^{d_context}) operations for epsilon <= 0.01 and delta <= 0.001
-- Current RLHF systems: 10^3-10^4 samples from homogeneous pools
-- Required for global representation: 10^7-10^8 samples
-- Three strategic relaxation pathways: constrain representativeness to ~30 core values, scope robustness narrowly, or accept super-polynomial costs