---
type: claim
domain: ai-alignment
description: "Evaluator selection via representative sampling or citizens assemblies produces more legitimate alignment targets than convenience platforms like MTurk or volunteer communities"
confidence: likely
source: "Conitzer et al. 2024, 'Social Choice Should Guide AI Alignment' (ICML 2024)"
created: 2026-03-11
secondary_domains: [collective-intelligence, mechanisms]
---

# Representative sampling or deliberative assemblies outperform convenience platforms for AI alignment input

The method of selecting evaluators for AI alignment feedback has first-order effects on whose values are encoded in the system. The Conitzer paper argues that **representative sampling** (demographically balanced selection) or **deliberative mechanisms** (citizens' assemblies, sortition-based panels) produce more legitimate alignment targets than convenience platforms like Amazon Mechanical Turk or volunteer communities.

The problem with convenience sampling:
1. **Selection bias**: MTurk workers, Reddit volunteers, and beta testers are not representative of the population that will use the AI
2. **Incentive misalignment**: Paid microtask workers optimize for throughput, not thoughtful evaluation
3. **Homogeneity**: Volunteer communities self-select for similar values and preferences

Representative sampling addresses (1) by ensuring demographic balance. Deliberative assemblies address (2) and (3) by creating conditions for informed, reflective judgment rather than quick reactions.

This connects to [[democratic alignment assemblies produce constitutions as effective as expert-designed ones while better representing diverse populations]]—the evidence suggests that representative groups, given time and information, produce alignment targets comparable to expert-designed ones while better reflecting population diversity.

The paper explicitly recommends: "Representative sampling or deliberative mechanisms (citizens' assemblies) rather than convenience platforms."

## Evidence

- Conitzer et al. (2024) position paper recommends representative sampling and deliberative assemblies
- Contrasts these with "convenience platforms" (implicitly MTurk, volunteer communities)
- Connects to broader social choice literature on evaluator selection

## Implementation Challenges

Representative sampling and deliberative assemblies are more expensive and slower than convenience platforms. This creates a trade-off between legitimacy and iteration speed during AI development. The paper does not resolve this tension but makes it explicit.

---

Relevant Notes:
- [[democratic alignment assemblies produce constitutions as effective as expert-designed ones while better representing diverse populations]]
- [[community-centred norm elicitation surfaces alignment targets materially different from developer-specified rules]]
- [[collective intelligence requires diversity as a structural precondition not a moral preference]]

Topics:
- domains/ai-alignment/_map
- foundations/collective-intelligence/_map