2f86a53bc8
- Fixed based on eval review comments - Quality gate pass 3 (fix-from-feedback) Pentagon-Agent: Theseus <HEADLESS>
4 KiB
| type | title | description | confidence | created | processed_date | source | enrichments | |
|---|---|---|---|---|---|---|---|---|
| claim | Egalitarian aggregation through minmax regret bounds worst case preference group dissatisfaction in pluralistic AI deployment | MinMax Regret aggregation provides an egalitarian mechanism for combining diverse preference groups by minimizing the maximum dissatisfaction any group experiences, operationalizing fairness through social choice theory | experimental | 2026-03-11 | 2026-03-11 | EM-DPO Heterogeneous Preferences Extraction (2025-00-00-em-dpo-heterogeneous-preferences-extraction) |
|
Egalitarian aggregation through minmax regret bounds worst case preference group dissatisfaction in pluralistic AI deployment
MinMax Regret aggregation provides a formal mechanism for combining outputs from multiple preference-aligned models while guaranteeing fairness across groups. The EM-DPO paper implements this as the deployment-time aggregation strategy after training K separate models on discovered preference types.
The mechanism:
- Train K separate models, each optimized for one latent preference type (discovered via EM algorithm)
- At inference, for each query, generate outputs from all K models
- Select the output that minimizes the maximum regret across groups: argmin_{output} max_{group} (regret_{group}(output))
Regret is defined as the difference between a group's utility for their preferred output versus the selected output. This ensures no single preference group experiences catastrophic dissatisfaction, even if it means average satisfaction is lower than a utilitarian aggregation would achieve.
Contrast with utilitarian aggregation:
Standard RLHF effectively implements utilitarian aggregation by maximizing average reward across all annotators. This can leave minority preference groups severely dissatisfied if their preferences conflict with the majority. MinMax Regret instead optimizes for the worst-off group, accepting lower average satisfaction to prevent extreme dissatisfaction for any group.
Connection to social choice theory:
MinMax Regret is a well-established mechanism in social choice theory and mechanism design. Arrow's Impossibility Theorem proved that no aggregation mechanism can satisfy all fairness criteria simultaneously (unanimity, non-dictatorship, independence of irrelevant alternatives, transitivity) when preferences genuinely diverge. MinMax Regret accepts this impossibility and instead optimizes for a specific fairness criterion: egalitarian worst-case protection. It explicitly trades off average satisfaction for bounded inequality.
The novelty is application, not mechanism: The theoretical foundations of MinMax Regret are proven in social choice theory. What's experimental is applying this aggregation strategy to LLM deployment with multiple preference-aligned models. The EM-DPO paper demonstrates this application empirically but doesn't claim to have invented the underlying mechanism.
Why this matters for pluralistic AI deployment:
In systems serving diverse populations with irreducible value differences, a single aggregated model will inevitably disappoint some groups severely. MinMax Regret operationalizes the principle that disagreements rooted in genuine value differences cannot be resolved through consensus by explicitly mapping preference diversity into system structure (ensemble of type-specific models) rather than attempting to collapse it into a single policy.
Relevant Notes:
- pluralistic-alignment-must-accommodate-irreducibly-diverse-values-simultaneously-rather-than-converging-on-a-single-aligned-state — MinMax Regret is a technical instantiation of this principle
- binary-preference-comparisons-cannot-identify-latent-preference-types-making-pairwise-RLHF-structurally-blind-to-diversity — EM-DPO's EM stage discovers the preference types that MinMax Regret then aggregates
Topics: AI alignment, social choice theory, fairness, preference aggregation, egalitarianism