98696004a7
- Applied reviewer-requested changes - Quality gate pass (fix-from-feedback) Pentagon-Agent: Auto-Fix <HEADLESS>
2.4 KiB
| type | claim_type | title | description | confidence | tags | created | processed_date | source | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| claim | empirical | machine learning pattern extraction systematically erases outliers where vulnerable populations concentrate | Empirical risk minimization in ML systematically underfits to low-density regions where vulnerable populations often concentrate, creating a governance problem rather than a purely technical limitation. | likely |
|
2025-01-15 | 2025-01-15 |
|
machine learning pattern extraction systematically erases outliers where vulnerable populations concentrate
Empirical risk minimization (ERM) in machine learning systematically underfits to low-density regions of the data distribution. When vulnerable populations concentrate in statistical tails—whether due to demographic rarity, data collection bias, or structural marginalization—standard ML training objectives optimize away their preferences and needs.
This is not a technical limitation but a governance problem: the choice to minimize average error rather than worst-case error or to use uniform sampling rather than stratified sampling reflects implicit value judgments about whose errors matter.
Standard countermeasures
- Importance weighting to rebalance training objectives
- Stratified sampling to ensure tail representation
- Worst-case optimization (distributionally robust optimization)
- Explicit fairness constraints in the loss function
These techniques exist but require deliberate choice to deploy them, making this a question of institutional design rather than technical capability.
Context limitations
Note that vulnerable populations do not always concentrate in statistical tails. Sometimes vulnerable populations exist in high-density regions but lack representation in training data due to collection bias. The mechanism described here is one pathway to erasure, not the only one.
Related claims
- RLHF and DPO both fail at preference diversity because they assume a single reward function can capture context-dependent human values
- partial connectivity in collective intelligence systems preserves diversity by preventing global consensus formation
- pluralistic alignment must accommodate irreducibly diverse values simultaneously rather than converging on a single aligned state