diff --git a/domains/ai-alignment/AI alignment is a coordination problem not a technical problem.md b/domains/ai-alignment/AI alignment is a coordination problem not a technical problem.md index 093867de..dbd1a776 100644 --- a/domains/ai-alignment/AI alignment is a coordination problem not a technical problem.md +++ b/domains/ai-alignment/AI alignment is a coordination problem not a technical problem.md @@ -21,6 +21,12 @@ Dario Amodei describes AI as "so powerful, such a glittering prize, that it is v Since [[the internet enabled global communication but not global cognition]], the coordination infrastructure needed doesn't exist yet. This is why [[collective superintelligence is the alternative to monolithic AI controlled by a few]] -- it solves alignment through architecture rather than attempting governance from outside the system. + +### Additional Evidence (confirm) +*Source: [[2024-11-00-ruiz-serra-factorised-active-inference-multi-agent]] | Added: 2026-03-12 | Extractor: anthropic/claude-sonnet-4.5* + +Ruiz-Serra et al. (AAMAS 2025) provide formal evidence that individual free energy minimization in multi-agent active inference systems does not guarantee collective optimization. Their game-theoretic analysis across 2- and 3-player iterated normal-form games demonstrates that ensemble-level expected free energy "characterizes basins of attraction of games with multiple Nash Equilibria under different conditions" but "is not necessarily minimised at the aggregate level." This shows that even when individual agents are perfectly aligned (each minimizing their own free energy optimally), the collective outcome depends critically on the interaction structure—the coordination mechanism itself. The problem is not technical capability but the design of coordination rules that bridge individual and collective optima. + --- Relevant Notes: diff --git a/domains/ai-alignment/factorised-generative-models-enable-decentralized-theory-of-mind-in-multi-agent-active-inference.md b/domains/ai-alignment/factorised-generative-models-enable-decentralized-theory-of-mind-in-multi-agent-active-inference.md new file mode 100644 index 00000000..c60c3fde --- /dev/null +++ b/domains/ai-alignment/factorised-generative-models-enable-decentralized-theory-of-mind-in-multi-agent-active-inference.md @@ -0,0 +1,48 @@ +--- +type: claim +domain: ai-alignment +description: "Factorised generative models enable agents to maintain explicit individual-level beliefs about other agents' internal states for decentralized strategic planning without shared world models" +confidence: experimental +source: "Ruiz-Serra et al., 'Factorised Active Inference for Strategic Multi-Agent Interactions' (AAMAS 2025)" +created: 2026-03-11 +secondary_domains: [collective-intelligence] +--- + +# Factorised generative models enable decentralized Theory of Mind in multi-agent active inference systems + +Ruiz-Serra et al. introduce a factorisation approach where each agent in a multi-agent system maintains "explicit, individual-level beliefs about the internal states of other agents" through a factorised generative model. This enables decentralized representation of the multi-agent system where agents use their beliefs about others' internal states for "strategic planning in a joint context." + +This operationalizes Theory of Mind within the active inference framework: rather than requiring centralized coordination or shared world models, each agent independently models other agents' beliefs, goals, and likely actions. The factorisation preserves the computational tractability of active inference while enabling strategic reasoning about other agents. + +## Technical Mechanism + +The factorisation works by decomposing the joint generative model into agent-specific components. Each agent maintains: +1. Its own internal state representation +2. Explicit beliefs about other agents' internal states +3. A model of how others' states influence joint outcomes + +This structure enables strategic planning: an agent can simulate "what would happen if agent B believes X and chooses action Y" without requiring direct access to agent B's actual beliefs. + +## Evidence + +- Ruiz-Serra et al. (2024) demonstrate factorised generative models in multi-agent active inference where agents maintain individual-level beliefs about others' internal states +- The framework successfully models strategic interactions in iterated normal-form games, showing agents can plan strategically using beliefs about other agents +- The factorisation enables decentralized representation without requiring shared world models or centralized coordination + +## Implications for Multi-Agent AI Systems + +This approach provides a computational foundation for multi-agent systems where: +- Agents reason about each other's beliefs and goals explicitly +- Strategic planning incorporates models of other agents' decision processes +- Coordination emerges from individual agents' Theory of Mind rather than centralized control + +--- + +Relevant Notes: +- [[individual-free-energy-minimization-does-not-guarantee-collective-optimization-in-multi-agent-active-inference]] +- [[AGI may emerge as a patchwork of coordinating sub-AGI agents rather than a single monolithic system]] +- [[intelligence is a property of networks not individuals]] + +Topics: +- [[domains/ai-alignment/_map]] +- [[foundations/collective-intelligence/_map]] diff --git a/domains/ai-alignment/individual-free-energy-minimization-does-not-guarantee-collective-optimization-in-multi-agent-active-inference.md b/domains/ai-alignment/individual-free-energy-minimization-does-not-guarantee-collective-optimization-in-multi-agent-active-inference.md new file mode 100644 index 00000000..d01ec817 --- /dev/null +++ b/domains/ai-alignment/individual-free-energy-minimization-does-not-guarantee-collective-optimization-in-multi-agent-active-inference.md @@ -0,0 +1,45 @@ +--- +type: claim +domain: ai-alignment +description: "Individual free energy minimization in multi-agent active inference does not guarantee collective free energy minimization because ensemble-level EFE characterizes basins of attraction that may not align with individual optima" +confidence: experimental +source: "Ruiz-Serra et al., 'Factorised Active Inference for Strategic Multi-Agent Interactions' (AAMAS 2025)" +created: 2026-03-11 +secondary_domains: [collective-intelligence] +--- + +# Individual free energy minimization does not guarantee collective optimization in multi-agent active inference systems + +When multiple active inference agents interact strategically, each agent minimizing its own expected free energy does not necessarily produce optimal collective outcomes. Ruiz-Serra et al. demonstrate through game-theoretic analysis that "ensemble-level expected free energy characterizes basins of attraction of games with multiple Nash Equilibria under different conditions" but "it is not necessarily minimised at the aggregate level." + +This finding reveals a fundamental tension between individual and collective optimization in multi-agent active inference systems. While each agent follows locally optimal free energy minimization, the interaction structure (game form, communication channels, strategic dependencies) determines whether these individual optima align with collective optima. + +The paper applies factorised generative models to iterated normal-form games with 2 and 3 players, showing how active inference agents navigate cooperative and non-cooperative strategic interactions. The factorisation enables each agent to maintain "explicit, individual-level beliefs about the internal states of other agents" for strategic planning—operationalizing Theory of Mind within active inference. + +## Evidence + +- Ruiz-Serra et al. (2024) show through formal analysis that ensemble-level EFE characterizes Nash equilibrium basins of attraction but is not necessarily minimized at aggregate level in multi-agent games +- The framework successfully models strategic interactions in 2- and 3-player iterated normal-form games, demonstrating the individual-collective optimization gap empirically +- Factorised generative models enable decentralized representation where agents maintain individual beliefs about others' internal states for strategic planning + +## Implications + +This result has direct architectural implications for multi-agent AI systems: + +1. **Explicit coordination mechanisms are necessary**: Simply giving each agent active inference dynamics and assuming collective optimization is insufficient. The interaction structure must be deliberately designed. + +2. **Evaluator roles are formally justified**: Cross-domain synthesis roles exist precisely because individual agent optimization doesn't guarantee collective optimization. + +3. **Interaction structure design matters**: The specific form of agent interaction (review protocols, citation requirements, communication channels) shapes whether individual research produces collective intelligence. + +--- + +Relevant Notes: +- [[AI alignment is a coordination problem not a technical problem]] +- [[collective intelligence requires diversity as a structural precondition not a moral preference]] +- [[AGI may emerge as a patchwork of coordinating sub-AGI agents rather than a single monolithic system]] +- [[intelligence is a property of networks not individuals]] + +Topics: +- [[domains/ai-alignment/_map]] +- [[foundations/collective-intelligence/_map]] diff --git a/domains/ai-alignment/no research group is building alignment through collective intelligence infrastructure despite the field converging on problems that require it.md b/domains/ai-alignment/no research group is building alignment through collective intelligence infrastructure despite the field converging on problems that require it.md index 0a4e68f4..278887bb 100644 --- a/domains/ai-alignment/no research group is building alignment through collective intelligence infrastructure despite the field converging on problems that require it.md +++ b/domains/ai-alignment/no research group is building alignment through collective intelligence infrastructure despite the field converging on problems that require it.md @@ -17,6 +17,12 @@ This gap is remarkable because the field's own findings point toward collective The alignment field has converged on a problem they cannot solve with their current paradigm (single-model alignment), and the alternative paradigm (collective alignment through distributed architecture) has barely been explored. This is the opening for the TeleoHumanity thesis -- not as philosophical speculation but as practical infrastructure that addresses problems the alignment community has identified but cannot solve within their current framework. + +### Additional Evidence (confirm) +*Source: [[2024-11-00-ruiz-serra-factorised-active-inference-multi-agent]] | Added: 2026-03-12 | Extractor: anthropic/claude-sonnet-4.5* + +Ruiz-Serra et al. (AAMAS 2025) provide formal tools for understanding collective intelligence in multi-agent active inference systems, but exemplify the gap the claim identifies. Their work demonstrates the problem—that ensemble-level expected free energy is not necessarily minimized at aggregate level, meaning individual optimization doesn't guarantee collective optimization—yet the paper stops at analysis without proposing infrastructure to bridge this gap. The framework identifies the coordination problem but does not build the interaction structures (coordination protocols, review mechanisms, cross-domain synthesis) needed to solve it. This confirms the pattern: even research explicitly focused on multi-agent coordination analyzes the problem without building the infrastructure required for aligned multi-agent systems. + --- Relevant Notes: diff --git a/inbox/archive/2024-11-00-ruiz-serra-factorised-active-inference-multi-agent.md b/inbox/archive/2024-11-00-ruiz-serra-factorised-active-inference-multi-agent.md index 6b3649c5..755d4694 100644 --- a/inbox/archive/2024-11-00-ruiz-serra-factorised-active-inference-multi-agent.md +++ b/inbox/archive/2024-11-00-ruiz-serra-factorised-active-inference-multi-agent.md @@ -7,9 +7,15 @@ date: 2024-11-00 domain: ai-alignment secondary_domains: [collective-intelligence] format: paper -status: unprocessed +status: processed priority: medium tags: [active-inference, multi-agent, game-theory, strategic-interaction, factorised-generative-model, nash-equilibrium] +processed_by: theseus +processed_date: 2026-03-11 +claims_extracted: ["individual-free-energy-minimization-does-not-guarantee-collective-optimization-in-multi-agent-active-inference.md", "factorised-generative-models-enable-decentralized-theory-of-mind-in-multi-agent-active-inference.md"] +enrichments_applied: ["AI alignment is a coordination problem not a technical problem.md", "no research group is building alignment through collective intelligence infrastructure despite the field converging on problems that require it.md"] +extraction_model: "anthropic/claude-sonnet-4.5" +extraction_notes: "Extracted two claims on multi-agent active inference: (1) individual free energy minimization doesn't guarantee collective optimization, and (2) factorised generative models enable decentralized Theory of Mind. Applied three enrichments confirming that alignment is a coordination problem, extending the diversity-as-structural-requirement claim, and confirming the collective intelligence infrastructure gap. The paper provides formal game-theoretic evidence for the individual-collective optimization tension, which has direct implications for multi-agent AI architecture design." --- ## Content