2385f134bc
- Source: inbox/archive/2024-04-00-conitzer-social-choice-guide-alignment.md - Domain: ai-alignment - Extracted by: headless extraction cron (worker 3) Pentagon-Agent: Theseus <HEADLESS>
2.9 KiB
| type | domain | secondary_domains | description | confidence | source | created | depends_on | challenged_by | ||
|---|---|---|---|---|---|---|---|---|---|---|
| claim | ai-alignment |
|
When values are genuinely incompatible, creating multiple aligned AI systems is structurally superior to aggregating into a single system | experimental | Conitzer et al. 2024 ICML position paper proposing pluralism as structural alternative to forced consensus | 2024-12-19 |
|
Pluralistic alignment creates multiple AI systems reflecting incompatible values rather than forcing consensus
When human values are genuinely incompatible—not merely diverse but irreducibly conflicting—the optimal alignment strategy may be to create multiple AI systems that reflect different value sets rather than aggregating all preferences into a single aligned system.
Conitzer et al. (2024) propose this "pluralism option" as a structural alternative to the standard alignment paradigm. The key insight: Some disagreements stem from fundamental value differences, not information gaps. Forcing these into a single aggregated preference function either:
- Imposes one group's values on others (creating a de facto dictatorship)
- Produces an incoherent compromise that satisfies no one
- Hides value conflicts behind technical aggregation choices
The pluralistic approach instead:
- Identifies clusters of genuinely incompatible values (e.g., different religious traditions, political philosophies, or cultural frameworks)
- Develops separate AI systems aligned to each cluster
- Allows users to choose which system to interact with based on their values
- Makes value conflicts explicit rather than obscuring them through aggregation
This aligns with the broader collective superintelligence thesis: rather than a single monolithic AI controlled by whoever wins the alignment race, a diverse ecosystem of aligned systems preserves human agency and value pluralism.
Practical implementation challenges:
- How to identify genuine value incompatibility vs. resolvable disagreement
- Whether to allow systems aligned to harmful value sets (and who decides what's harmful)
- How to handle interactions between users of different systems
- Resource allocation when developing multiple systems is more expensive than one
The paper does not fully resolve these challenges but establishes pluralism as a legitimate structural option rather than a failure mode. This represents a significant departure from the "solve alignment once" framing that dominates the field.
Relevant Notes:
- collective superintelligence is the alternative to monolithic AI controlled by a few
- pluralistic alignment must accommodate irreducibly diverse values simultaneously rather than converging on a single aligned state
- persistent irreducible disagreement.md
- AI alignment is a coordination problem not a technical problem
Topics: