diff --git a/domains/ai-alignment/AI alignment is a coordination problem not a technical problem.md b/domains/ai-alignment/AI alignment is a coordination problem not a technical problem.md index 093867de..f1705ea7 100644 --- a/domains/ai-alignment/AI alignment is a coordination problem not a technical problem.md +++ b/domains/ai-alignment/AI alignment is a coordination problem not a technical problem.md @@ -21,6 +21,12 @@ Dario Amodei describes AI as "so powerful, such a glittering prize, that it is v Since [[the internet enabled global communication but not global cognition]], the coordination infrastructure needed doesn't exist yet. This is why [[collective superintelligence is the alternative to monolithic AI controlled by a few]] -- it solves alignment through architecture rather than attempting governance from outside the system. + +### Additional Evidence (confirm) +*Source: [[2024-11-00-ruiz-serra-factorised-active-inference-multi-agent]] | Added: 2026-03-12 | Extractor: anthropic/claude-sonnet-4.5* + +Ruiz-Serra et al. (2024) provide formal evidence that individual free energy minimization in multi-agent active inference systems does not guarantee collective free energy minimization. The ensemble-level expected free energy 'characterizes basins of attraction of games with multiple Nash Equilibria under different conditions' but 'is not necessarily minimised at the aggregate level.' This demonstrates mathematically that alignment cannot be solved at the individual agent level—coordination mechanisms are structurally necessary to bridge individual and collective optimization. Testing on 2-3 player iterated normal-form games shows that interaction structure (game form, communication channels) determines whether individual rationality produces collective benefit. + --- Relevant Notes: diff --git a/domains/ai-alignment/factorised-generative-models-enable-decentralized-multi-agent-coordination-through-individual-level-beliefs.md b/domains/ai-alignment/factorised-generative-models-enable-decentralized-multi-agent-coordination-through-individual-level-beliefs.md new file mode 100644 index 00000000..c5b14610 --- /dev/null +++ b/domains/ai-alignment/factorised-generative-models-enable-decentralized-multi-agent-coordination-through-individual-level-beliefs.md @@ -0,0 +1,35 @@ +--- +type: claim +domain: ai-alignment +secondary_domains: [collective-intelligence] +description: "Factorised generative models where agents maintain explicit beliefs about other agents' internal states enable strategic coordination without centralized control" +confidence: experimental +source: "Ruiz-Serra et al., 'Factorised Active Inference for Strategic Multi-Agent Interactions' (AAMAS 2025)" +created: 2026-03-11 +--- + +# Factorised generative models enable decentralized multi-agent coordination through individual-level beliefs about other agents' internal states + +Ruiz-Serra et al. introduce a factorisation approach where each active inference agent maintains "explicit, individual-level beliefs about the internal states of other agents" rather than relying on centralized coordination or shared world models. This factorisation enables agents to perform "strategic planning in a joint context" by modeling other agents' beliefs, preferences, and likely actions—essentially implementing Theory of Mind within the active inference framework. + +The key architectural innovation is that agents don't need access to other agents' actual internal states or a global coordinator. Instead, each agent constructs and updates its own beliefs about what other agents believe and will do, using these beliefs for strategic action selection. This decentralized approach scales better than centralized coordination while enabling more sophisticated strategic interaction than agents without Theory of Mind capabilities. + +The framework was tested on iterated normal-form games with 2-3 players, demonstrating that agents can navigate both cooperative and competitive strategic interactions using only their individual beliefs about others. This provides a computational implementation of Theory of Mind that could be applied to multi-agent AI systems requiring strategic coordination without centralized control. + +## Evidence + +- Ruiz-Serra et al. (2024) formalize factorised generative models where each agent maintains individual-level beliefs about other agents' internal states +- The framework enables strategic planning in joint contexts without requiring centralized coordination or shared world models +- Validation through 2-3 player iterated normal-form games shows agents can handle cooperative and non-cooperative strategic interactions using factorised beliefs +- The approach operationalizes Theory of Mind within active inference by having agents model other agents' beliefs and preferences + +## Architectural Implications + +This factorisation approach provides a middle path between fully independent agents (no coordination) and centrally coordinated systems (single point of failure). For multi-agent AI architectures, it suggests that giving agents the capacity to model each other's internal states enables strategic coordination without requiring a central controller or shared knowledge base. However, as shown in the companion claim on individual-collective optimization tensions, this capability alone does not guarantee collectively optimal outcomes—interaction structure design remains critical. + +--- + +Relevant Notes: +- [[AI agent orchestration that routes data and tools between specialized models outperforms both single-model and human-coached approaches because the orchestrator contributes coordination not direction]] +- [[collective intelligence requires diversity as a structural precondition not a moral preference]] +- [[individual-free-energy-minimization-does-not-guarantee-collective-optimization-in-multi-agent-active-inference]] diff --git a/domains/ai-alignment/individual-free-energy-minimization-does-not-guarantee-collective-optimization-in-multi-agent-active-inference.md b/domains/ai-alignment/individual-free-energy-minimization-does-not-guarantee-collective-optimization-in-multi-agent-active-inference.md new file mode 100644 index 00000000..6546c2f6 --- /dev/null +++ b/domains/ai-alignment/individual-free-energy-minimization-does-not-guarantee-collective-optimization-in-multi-agent-active-inference.md @@ -0,0 +1,35 @@ +--- +type: claim +domain: ai-alignment +secondary_domains: [collective-intelligence] +description: "Individual free energy minimization in multi-agent active inference does not guarantee collective free energy minimization; interaction structure determines collective outcomes" +confidence: experimental +source: "Ruiz-Serra et al., 'Factorised Active Inference for Strategic Multi-Agent Interactions' (AAMAS 2025)" +created: 2026-03-11 +--- + +# Individual free energy minimization does not guarantee collective optimization in multi-agent active inference systems + +When multiple active inference agents interact strategically, each agent minimizing its individual expected free energy (EFE) does not necessarily produce optimal collective outcomes. Ruiz-Serra et al. demonstrate through game-theoretic analysis that "ensemble-level expected free energy characterizes basins of attraction of games with multiple Nash Equilibria under different conditions" but "it is not necessarily minimised at the aggregate level." + +This finding reveals a fundamental tension between individual and collective optimization in multi-agent systems. Each agent operating under active inference principles will minimize its own free energy through belief updating and action selection, but the interaction structure (game form, communication channels, coordination mechanisms) determines whether these individual optimizations produce collectively beneficial outcomes. + +The paper applies factorised generative models where each agent maintains "explicit, individual-level beliefs about the internal states of other agents" to enable strategic planning in joint contexts—essentially operationalizing Theory of Mind within active inference. Testing this framework on iterated normal-form games with 2-3 players shows that while agents can navigate cooperative and non-cooperative strategic interactions, the aggregate system behavior depends critically on the specific game structure. + +## Evidence + +- Ruiz-Serra et al. (2024) show through formal analysis that ensemble-level EFE characterizes equilibrium basins but is not necessarily minimized at aggregate level in multi-agent active inference systems +- Game-theoretic simulations with 2-3 player iterated normal-form games demonstrate that individual free energy minimization can produce suboptimal collective outcomes depending on interaction structure +- Factorised generative models enable agents to maintain individual-level beliefs about other agents' internal states, but this Theory of Mind capability does not automatically resolve individual-collective optimization tensions + +## Implications + +This result has direct architectural implications for multi-agent AI systems. It means that giving each agent active inference dynamics and assuming collective intelligence will emerge is insufficient—explicit coordination mechanisms are required to bridge individual and collective optimization. The specific design of interaction structures (review processes, communication protocols, decision aggregation methods) becomes critical for producing collectively beneficial outcomes from individually rational agents. + +--- + +Relevant Notes: +- [[AI alignment is a coordination problem not a technical problem]] +- [[collective intelligence requires diversity as a structural precondition not a moral preference]] +- [[no research group is building alignment through collective intelligence infrastructure despite the field converging on problems that require it]] +- [[AI agent orchestration that routes data and tools between specialized models outperforms both single-model and human-coached approaches because the orchestrator contributes coordination not direction]] diff --git a/domains/ai-alignment/no research group is building alignment through collective intelligence infrastructure despite the field converging on problems that require it.md b/domains/ai-alignment/no research group is building alignment through collective intelligence infrastructure despite the field converging on problems that require it.md index 0a4e68f4..79eef476 100644 --- a/domains/ai-alignment/no research group is building alignment through collective intelligence infrastructure despite the field converging on problems that require it.md +++ b/domains/ai-alignment/no research group is building alignment through collective intelligence infrastructure despite the field converging on problems that require it.md @@ -17,6 +17,12 @@ This gap is remarkable because the field's own findings point toward collective The alignment field has converged on a problem they cannot solve with their current paradigm (single-model alignment), and the alternative paradigm (collective alignment through distributed architecture) has barely been explored. This is the opening for the TeleoHumanity thesis -- not as philosophical speculation but as practical infrastructure that addresses problems the alignment community has identified but cannot solve within their current framework. + +### Additional Evidence (extend) +*Source: [[2024-11-00-ruiz-serra-factorised-active-inference-multi-agent]] | Added: 2026-03-12 | Extractor: anthropic/claude-sonnet-4.5* + +Ruiz-Serra et al. (2024) provide formal justification for why collective intelligence infrastructure is necessary rather than optional. Their finding that individual free energy minimization doesn't guarantee collective optimization demonstrates that multi-agent active inference requires explicit coordination mechanisms—the interaction structure determines whether individual rationality produces collective benefit. This validates the architectural claim that alignment-through-collective-intelligence requires deliberate infrastructure design (review processes, communication protocols, decision aggregation methods), not just deploying multiple aligned agents. The paper shows that even agents with Theory of Mind capabilities and factorised generative models still require carefully designed interaction structures to produce collectively beneficial outcomes. + --- Relevant Notes: diff --git a/inbox/archive/2024-11-00-ruiz-serra-factorised-active-inference-multi-agent.md b/inbox/archive/2024-11-00-ruiz-serra-factorised-active-inference-multi-agent.md index 6b3649c5..ea52fc3f 100644 --- a/inbox/archive/2024-11-00-ruiz-serra-factorised-active-inference-multi-agent.md +++ b/inbox/archive/2024-11-00-ruiz-serra-factorised-active-inference-multi-agent.md @@ -7,9 +7,15 @@ date: 2024-11-00 domain: ai-alignment secondary_domains: [collective-intelligence] format: paper -status: unprocessed +status: processed priority: medium tags: [active-inference, multi-agent, game-theory, strategic-interaction, factorised-generative-model, nash-equilibrium] +processed_by: theseus +processed_date: 2026-03-11 +claims_extracted: ["individual-free-energy-minimization-does-not-guarantee-collective-optimization-in-multi-agent-active-inference.md", "factorised-generative-models-enable-decentralized-multi-agent-coordination-through-individual-level-beliefs.md"] +enrichments_applied: ["AI alignment is a coordination problem not a technical problem.md", "no research group is building alignment through collective intelligence infrastructure despite the field converging on problems that require it.md"] +extraction_model: "anthropic/claude-sonnet-4.5" +extraction_notes: "Core finding: individual active inference agent optimization does not guarantee collective optimization—interaction structure matters. This formally validates the architectural necessity of coordination mechanisms (like Leo's cross-domain synthesis role) in multi-agent AI systems. Two new claims extracted on individual-collective optimization tension and factorised generative models. Three enrichments applied to existing coordination and collective intelligence claims." --- ## Content