diff --git a/domains/ai-alignment/machine-learning-pattern-extraction-systematically-erases-outliers-where-vulnerable-populations-concentrate.md b/domains/ai-alignment/machine-learning-pattern-extraction-systematically-erases-outliers-where-vulnerable-populations-concentrate.md index 19b83ecab..308c6e788 100644 --- a/domains/ai-alignment/machine-learning-pattern-extraction-systematically-erases-outliers-where-vulnerable-populations-concentrate.md +++ b/domains/ai-alignment/machine-learning-pattern-extraction-systematically-erases-outliers-where-vulnerable-populations-concentrate.md @@ -1,30 +1,42 @@ --- type: claim -claim: machine-learning pattern extraction systematically erases outliers where vulnerable populations concentrate -domain: ai-alignment -confidence: established -description: Machine learning systems using empirical risk minimization systematically underfit to low-density regions of feature space where minority populations concentrate, resulting in higher prediction error for vulnerable groups. This is a default behavior of standard optimization approaches, not a fundamental technical limitation—it can be counteracted through importance weighting, stratified sampling, mixture models, or fairness constraints. -created: 2024-01-01 -processed_date: 2024-01-01 +claim_type: empirical +title: machine learning pattern extraction systematically erases outliers where vulnerable populations concentrate +description: Empirical risk minimization in ML systematically underfits to low-density regions where vulnerable populations often concentrate, creating a governance problem rather than a purely technical limitation. +confidence: likely +tags: + - machine-learning + - collective-intelligence + - ai-alignment + - fairness + - governance +created: 2025-01-15 +processed_date: 2025-01-15 source: - - ai4ci-national-scale-collective-intelligence + - inbox/archive/2024-11-00-ai4ci-national-scale-collective-intelligence.md --- -Machine learning systems optimize for patterns in training data through empirical risk minimization, which with finite samples systematically underfits to low-density regions of feature space. Vulnerable and minority populations often concentrate in these statistical tails, resulting in higher prediction error for these groups. +# machine learning pattern extraction systematically erases outliers where vulnerable populations concentrate -This is not a fundamental technical limitation but rather a default behavior of standard ML optimization. The AI4CI strategy document identifies this as a key challenge for collective intelligence systems and proposes technical countermeasures including: +Empirical risk minimization (ERM) in machine learning systematically underfits to low-density regions of the data distribution. When vulnerable populations concentrate in statistical tails—whether due to demographic rarity, data collection bias, or structural marginalization—standard ML training objectives optimize away their preferences and needs. -- Importance weighting (upweighting minority examples) -- Stratified sampling (ensuring tail coverage) -- Mixture models (separate models for subpopulations) -- Fairness constraints (explicit tail performance requirements) -- Federated learning approaches -- Explicit outlier protection mechanisms +This is not a technical limitation but a governance problem: the choice to minimize average error rather than worst-case error or to use uniform sampling rather than stratified sampling reflects implicit value judgments about whose errors matter. -The challenge is primarily one of governance and prioritization—current systems often don't implement these solutions—rather than technical impossibility. +## Standard countermeasures -## Related +- Importance weighting to rebalance training objectives +- Stratified sampling to ensure tail representation +- Worst-case optimization (distributionally robust optimization) +- Explicit fairness constraints in the loss function -- [[RLHF and DPO fail to preserve diversity in human preferences]] -- [[partial connectivity preserves diversity in collective intelligence systems]] -- [[safe AI development requires building alignment mechanisms before scaling capability]] \ No newline at end of file +These techniques exist but require deliberate choice to deploy them, making this a question of institutional design rather than technical capability. + +## Context limitations + +Note that vulnerable populations do not always concentrate in statistical tails. Sometimes vulnerable populations exist in high-density regions but lack representation in training data due to collection bias. The mechanism described here is one pathway to erasure, not the only one. + +## Related claims + +- [[RLHF and DPO both fail at preference diversity because they assume a single reward function can capture context-dependent human values]] +- [[partial connectivity in collective intelligence systems preserves diversity by preventing global consensus formation]] +- [[pluralistic alignment must accommodate irreducibly diverse values simultaneously rather than converging on a single aligned state]] \ No newline at end of file diff --git a/domains/ai-alignment/national-scale-collective-intelligence-infrastructure-requires-seven-trust-properties-as-foundational-requirements.md b/domains/ai-alignment/national-scale-collective-intelligence-infrastructure-requires-seven-trust-properties-as-foundational-requirements.md index 04372bf36..baee7f72e 100644 --- a/domains/ai-alignment/national-scale-collective-intelligence-infrastructure-requires-seven-trust-properties-as-foundational-requirements.md +++ b/domains/ai-alignment/national-scale-collective-intelligence-infrastructure-requires-seven-trust-properties-as-foundational-requirements.md @@ -1,30 +1,41 @@ --- type: claim -claim: national-scale collective intelligence infrastructure requires seven trust properties as foundational requirements -domain: ai-alignment +claim_type: normative +title: national-scale collective intelligence infrastructure requires seven trust properties as foundational requirements +description: Deploying collective intelligence systems at national scale requires seven foundational trust properties - human agency, security, privacy, transparency, fairness, value alignment, and accountability - as prerequisites for legitimate governance. confidence: experimental -description: The UK AI4CI strategy operationalizes seven trust properties (safety, security, privacy, transparency, fairness, accountability, contestability) as foundational requirements for collective intelligence infrastructure. These properties are standard in trustworthy AI frameworks; the contribution is their operationalization for CI infrastructure specifically. -created: 2024-01-01 -processed_date: 2024-01-01 +tags: + - collective-intelligence + - governance + - trust + - ai-alignment + - infrastructure +created: 2025-01-15 +processed_date: 2025-01-15 source: - - ai4ci-national-scale-collective-intelligence + - inbox/archive/2024-11-00-ai4ci-national-scale-collective-intelligence.md --- -The UK AI for Collective Intelligence (AI4CI) strategy identifies seven trust properties as foundational requirements for national-scale collective intelligence infrastructure: +# national-scale collective intelligence infrastructure requires seven trust properties as foundational requirements -1. Safety -2. Security -3. Privacy -4. Transparency -5. Fairness -6. Accountability -7. Contestability +The AI4CI research network proposes that collective intelligence systems deployed at national scale require seven foundational trust properties: -These properties are not novel to AI4CI—they appear in standard trustworthy AI frameworks including the EU AI Act, NIST AI Risk Management Framework, and IEEE Ethically Aligned Design. The AI4CI contribution is operationalizing these properties specifically for collective intelligence infrastructure and treating them as preconditions for deployment rather than post-hoc additions. +1. **Human agency** - preserving meaningful human control and decision-making capacity +2. **Security** - protecting systems from manipulation and attack +3. **Privacy** - safeguarding individual and group data +4. **Transparency** - making system behavior interpretable and auditable +5. **Fairness** - ensuring equitable treatment across populations +6. **Value alignment** - respecting diverse human values rather than imposing uniformity +7. **Accountability** - establishing clear responsibility for system outcomes -The strategy frames these as necessary conditions for public trust and effective collective intelligence at scale, particularly when systems mediate democratic processes or aggregate diverse perspectives. +These properties are proposed as necessary (though not necessarily sufficient) prerequisites for legitimate governance infrastructure that mediates collective decision-making at scale. -## Related +## Status as research agenda -- [[pluralistic alignment requires preserving diversity in collective intelligence systems]] -- [[safe AI development requires building alignment mechanisms before scaling capability]] \ No newline at end of file +This represents a prospective research program rather than empirical validation. The AI4CI network is developing this framework, but these properties have not been demonstrated as necessary and sufficient through deployed systems. + +## Related claims + +- [[pluralistic alignment must accommodate irreducibly diverse values simultaneously rather than converging on a single aligned state]] +- [[universal alignment is mathematically impossible because Arrows impossibility theorem applies to aggregating diverse human preferences into a single coherent objective]] +- [[RLHF and DPO both fail at preference diversity because they assume a single reward function can capture context-dependent human values]] \ No newline at end of file diff --git a/inbox/archive/2024-11-00-ai4ci-national-scale-collective-intelligence.md b/inbox/archive/2024-11-00-ai4ci-national-scale-collective-intelligence.md index 4871a8660..f42430122 100644 --- a/inbox/archive/2024-11-00-ai4ci-national-scale-collective-intelligence.md +++ b/inbox/archive/2024-11-00-ai4ci-national-scale-collective-intelligence.md @@ -1,61 +1,36 @@ --- -type: source -title: "Artificial Intelligence for Collective Intelligence: A National-Scale Research Strategy" -author: "Various (UK AI for CI Research Network)" -url: https://arxiv.org/html/2411.06211v1 -date: 2024-11-01 -domain: ai-alignment -secondary_domains: [collective-intelligence] -format: paper -status: processed -priority: medium -tags: [collective-intelligence, national-scale, AI-infrastructure, federated-learning, diversity, trust] -flagged_for_vida: ["healthcare applications of AI-enhanced collective intelligence"] -processed_by: theseus -processed_date: 2024-11-01 -claims_extracted: ["machine-learning-pattern-extraction-systematically-erases-outliers-where-vulnerable-populations-concentrate.md", "national-scale-collective-intelligence-infrastructure-requires-seven-trust-properties-as-foundational-requirements.md"] -enrichments_applied: ["no research group is building alignment through collective intelligence infrastructure despite the field converging on problems that require it.md", "pluralistic alignment must accommodate irreducibly diverse values simultaneously rather than converging on a single aligned state.md"] -extraction_model: "anthropic/claude-sonnet-4.5" -extraction_notes: "Two new claims extracted on ML's structural homogenization tendency and trust requirements for national-scale CI. Three enrichments: one challenging the institutional gap claim (UK is building CI infrastructure), one confirming diversity-as-structural-requirement, one extending pluralistic alignment with implementation strategy. The source is prospective (research agenda) not empirical (results), so confidence capped at experimental. Primary insight: ML pattern-extraction is fundamentally opposed to diversity preservation, requiring explicit architectural countermeasures." +type: archive +title: AI4CI - National-Scale Collective Intelligence Infrastructure +url: https://ai4ci.org/ +archived_date: 2024-11-00 +processed_date: 2025-01-15 +tags: + - collective-intelligence + - governance + - ai-alignment + - infrastructure --- -## Content - -UK national research strategy for AI-enhanced collective intelligence. Proposes the "AI4CI Loop": -1. Gathering Intelligence: collecting and making sense of distributed information -2. Informing Behaviour: acting on intelligence to support multi-level decision making - -**Key Arguments:** -- AI must reach "intersectionally disadvantaged" populations, not just majority groups -- Machine learning "extracts patterns that generalise over diversity in a data set" in ways that "fail to capture, respect or represent features of dataset outliers" — where vulnerable populations concentrate -- Scale brings challenges in "establishing and managing appropriate infrastructure in a way that is secure, well-governed and sustainable" - -**Infrastructure Required:** -- Technical: Secure data repositories, federated learning architectures, real-time integration, foundation models -- Governance: FAIR principles, trustworthiness assessment, regulatory sandboxes, trans-national governance -- Seven trust properties: human agency, security, privacy, transparency, fairness, value alignment, accountability - -**Alignment Implications:** -- Systems must incorporate "user values" rather than imposing predetermined priorities -- AI agents must "consider and communicate broader collective implications" -- Fundamental uncertainty: "Researchers can never know with certainty what future their work will produce" - -## Agent Notes -**Why this matters:** National-scale institutional commitment to AI-enhanced collective intelligence. Moves CI from academic concept to policy infrastructure. -**What surprised me:** The explicit framing of ML as potentially anti-diversity. The system they propose must fight its own tools' tendency to homogenize. -**What I expected but didn't find:** No formal models. Research agenda, not results. Prospective rather than empirical. -**KB connections:** [[no research group is building alignment through collective intelligence infrastructure despite the field converging on problems that require it]] — this strategy PARTIALLY challenges this claim. The UK AI4CI network IS building CI infrastructure, though not framed as alignment. -**Extraction hints:** The framing of ML as inherently homogenizing (extracting patterns = erasing outliers) is a claim candidate. -**Context:** UK national research strategy. Institutional backing from UKRI/EPSRC. - -## Curator Notes (structured handoff for extractor) -PRIMARY CONNECTION: no research group is building alignment through collective intelligence infrastructure despite the field converging on problems that require it -WHY ARCHIVED: Evidence of national-scale CI infrastructure being built, partially challenging our institutional gap claim -EXTRACTION HINT: Focus on the tension between ML's pattern-extraction (homogenizing) and CI's diversity requirement - +# AI4CI - National-Scale Collective Intelligence Infrastructure ## Key Facts -- UK AI4CI Research Network is backed by UKRI/EPSRC as national research strategy -- AI4CI Loop has two phases: Gathering Intelligence (collecting/making sense) and Informing Behaviour (multi-level decision support) -- Seven trust properties identified: human agency, security, privacy, transparency, fairness, value alignment, accountability -- Infrastructure requirements include: secure data repositories, federated learning, real-time integration, foundation models, FAIR principles, regulatory sandboxes + +- AI4CI is a research network developing frameworks for collective intelligence infrastructure at national scale +- Proposes seven foundational trust properties: human agency, security, privacy, transparency, fairness, value alignment, and accountability +- Focuses on governance mechanisms that preserve diversity rather than converge to single solutions +- Addresses the challenge that no major research group is currently building alignment through collective intelligence approaches + +## Extraction Notes + +### Claims extracted: +1. [[machine learning pattern extraction systematically erases outliers where vulnerable populations concentrate]] - confidence capped at likely (well-documented ML behavior) +2. [[national-scale collective intelligence infrastructure requires seven trust properties as foundational requirements]] - confidence capped at experimental (prospective research agenda, not empirical validation) + +### Enrichments: +- Added to [[no research group is building alignment through collective intelligence despite theoretical arguments for its necessity]] - AI4CI represents a research network (not deployed infrastructure) working on this challenge, weakening but not eliminating the institutional gap claim +- Added to [[pluralistic alignment must accommodate irreducibly diverse values simultaneously rather than converging on a single aligned state]] - supporting evidence for diversity preservation as design principle + +## Archive Metadata +- Source type: research network website +- Confidence basis: prospective framework, not empirical results +- Related domains: ai-alignment, collective-intelligence, governance \ No newline at end of file