---
type: claim
domain: ai-alignment
description: "Empirical study with 1,095 participants shows whose feedback trains the model matters as much as how much feedback"
confidence: likely
source: "arXiv 2511.14476, 27,375 ratings from 1,095 participants"
created: 2026-03-11
---

# Demographic composition of alignment training data produces measurable behavior differences of 3-5 percentage points

Systematic variation in the demographic composition of human feedback used for LLM alignment produces quantitatively significant differences in model behavior. This is the first large-scale empirical study (N=1,095 participants, 27,375 ratings) to systematically vary demographic composition while holding technical design constant.

## Evidence

Models fine-tuned on feedback from different demographic groups showed consistent behavioral divergence:
- Models fine-tuned on Liberal feedback: +5.0 percentage points vs Conservative baseline
- Models fine-tuned on White feedback: +4.7 percentage points vs Black baseline  
- Models fine-tuned on Female feedback: +3.4 percentage points vs Male baseline
- Effects measured on emotional awareness and toxicity dimensions

The magnitude of the effect—3-5 percentage points from demographic composition alone—is comparable to many technical design choices in alignment. This demonstrates that "whose preferences" is not merely a fairness concern but a quantitatively important variable in alignment outcomes, independent of the alignment technique used.

## Significance

The study jointly varied demographic composition and technical design, providing empirical evidence that the composition of the training population materially affects model behavior. This challenges the assumption that alignment can be achieved through a single training process applied uniformly across deployment contexts.

---

Relevant Notes:
- [[community-centred norm elicitation surfaces alignment targets materially different from developer-specified rules]]
- [[some disagreements are permanently irreducible because they stem from genuine value differences not information gaps and systems must map rather than eliminate them]]
- [[pluralistic alignment must accommodate irreducibly diverse values simultaneously rather than converging on a single aligned state]]

Topics:
- [[domains/ai-alignment/_map]]
