2.6 KiB
| type | domain | description | confidence | source | created | secondary_domains | |
|---|---|---|---|---|---|---|---|
| claim | ai-alignment | ML's core function of generalizing over diversity creates structural bias against marginalized groups | experimental | UK AI for CI Research Network, Artificial Intelligence for Collective Intelligence: A National-Scale Research Strategy (2024) | 2026-03-11 |
|
Machine learning pattern extraction systematically erases dataset outliers where vulnerable populations concentrate
Machine learning fundamentally "extracts patterns that generalise over diversity in a data set" in ways that "fail to capture, respect or represent features of dataset outliers." This is not a correctable bias but an inherent property of the optimization process. The UK AI4CI research strategy identifies this as a core tension: the same mechanism that makes ML powerful (finding generalizable patterns) makes it structurally hostile to diversity preservation.
The strategy explicitly frames this as requiring AI systems to reach "intersectionally disadvantaged" populations rather than optimizing for majority groups. Vulnerable populations concentrate in the statistical tails that pattern-extraction algorithms treat as noise to be smoothed away.
This creates a fundamental conflict for collective intelligence systems: ML tools are needed for scale, but their core function works against the diversity that collective intelligence requires.
Evidence
From the UK national research strategy:
- ML "extracts patterns that generalise over diversity in a data set" in ways that "fail to capture, respect or represent features of dataset outliers"
- Systems must reach "intersectionally disadvantaged" populations, not just majority groups
- The strategy identifies this as requiring infrastructure that fights against its own tools' homogenizing tendency
Relationship to Collective Intelligence
This claim directly challenges the feasibility of AI-enhanced collective intelligence at scale. If the tools required for scale (ML) systematically erase the diversity required for collective intelligence, then scaling collective intelligence through AI may be structurally impossible without fundamentally different architectures.
Relevant Notes:
- collective intelligence requires diversity as a structural precondition not a moral preference
- RLHF and DPO both fail at preference diversity because they assume a single reward function can capture context-dependent human values
- foundations/collective-intelligence/_map
Topics:
- domains/ai-alignment/_map
- foundations/collective-intelligence/_map