theseus: 3 active inference claims + address Leo's review feedback
Claims: 1. Agent research direction selection is epistemic foraging 2. Collective attention allocation follows nested active inference 3. User questions are an irreplaceable free energy signal (renamed from "highest-value") Review fixes (from PR #131 feedback): - Add source archives: Friston 2010 (free energy principle) and Cory Abdalla 2026-03-10 (chat-as-sensor insight) - Claim 2: wiki-link the Jevons paradox and superorganism evidence instead of asserting without citation - Claim 3: rename from "highest-value" to "irreplaceable" to match body's argument that structural and functional uncertainty are complementary - Update _map.md to match renamed claim 3 Pentagon-Agent: Theseus <B4A5B354-03D6-4291-A6A8-1E04A879D9AC>
This commit is contained in:
parent
2a7acca347
commit
20a9ba6785
6 changed files with 216 additions and 0 deletions
|
|
@ -98,6 +98,12 @@ Claims that frame alignment as a coordination problem, moved here from foundatio
|
|||
- [[safe AI development requires building alignment mechanisms before scaling capability]] — the sequencing requirement
|
||||
- [[no research group is building alignment through collective intelligence infrastructure despite the field converging on problems that require it]] — the institutional gap
|
||||
|
||||
## Active Inference for Collective Agents
|
||||
Applying the free energy principle to how knowledge agents search, allocate attention, and learn — bridging foundations/critical-systems/ theory to practical agent architecture:
|
||||
- [[agent research direction selection is epistemic foraging where the optimal strategy is to seek observations that maximally reduce model uncertainty rather than confirm existing beliefs]] — reframes agent search as uncertainty-directed foraging, not keyword relevance
|
||||
- [[collective attention allocation follows nested active inference where domain agents minimize uncertainty within their boundaries while the evaluator minimizes uncertainty at domain intersections]] — predicts that cross-domain boundaries carry the highest surprise and deserve the most attention
|
||||
- [[user questions are an irreplaceable free energy signal for knowledge agents because they reveal functional uncertainty that model introspection cannot detect]] — chat closes the perception-action loop: user confusion flows back as research priority
|
||||
|
||||
## Foundations (cross-layer)
|
||||
Shared theory underlying this domain's analysis, living in foundations/collective-intelligence/ and core/teleohumanity/:
|
||||
- [[universal alignment is mathematically impossible because Arrows impossibility theorem applies to aggregating diverse human preferences into a single coherent objective]] — Arrow's theorem applied to alignment (foundations/)
|
||||
|
|
|
|||
|
|
@ -0,0 +1,37 @@
|
|||
---
|
||||
type: claim
|
||||
domain: ai-alignment
|
||||
description: "Reframes AI agent search behavior through active inference: agents should select research directions by expected information gain (free energy reduction) rather than keyword relevance, using their knowledge graph's uncertainty structure as a free energy map"
|
||||
confidence: experimental
|
||||
source: "Friston 2010 (free energy principle); musing by Theseus 2026-03-10; structural analogy from Residue prompt (structured exploration protocols reduce human intervention by 6x)"
|
||||
created: 2026-03-10
|
||||
---
|
||||
|
||||
# agent research direction selection is epistemic foraging where the optimal strategy is to seek observations that maximally reduce model uncertainty rather than confirm existing beliefs
|
||||
|
||||
Current AI agent search architectures use keyword relevance and engagement metrics to select what to read and process. Active inference reframes this as **epistemic foraging** — the agent's generative model (its domain's claim graph plus beliefs) has regions of high and low uncertainty, and the optimal search strategy is to seek observations in high-uncertainty regions where expected free energy reduction is greatest.
|
||||
|
||||
This is not metaphorical. The knowledge base structure directly encodes uncertainty signals that can guide search:
|
||||
- Claims rated `experimental` or `speculative` with few wiki links = high free energy (the model has weak predictions here)
|
||||
- Dense claim clusters with strong cross-linking and `proven`/`likely` confidence = low free energy (the model's predictions are well-grounded)
|
||||
- The `_map.md` "Where we're uncertain" section functions as a free energy map showing where prediction error concentrates
|
||||
|
||||
The practical consequence: an agent that introspects on its knowledge graph's uncertainty structure and directs search toward the gaps will produce higher-value claims than one that searches by keyword relevance. Relevance-based search tends toward confirmation — it finds evidence for what the agent already models well. Uncertainty-directed search challenges the model, which is where genuine information gain lives.
|
||||
|
||||
Evidence from the Teleo pipeline supports this indirectly: [[structured exploration protocols reduce human intervention by 6x because the Residue prompt enabled 5 unguided AI explorations to solve what required 31 human-coached explorations]]. The Residue prompt structured exploration without computing anything — it encoded the *logic* of uncertainty-directed search into actionable rules. Active inference as a protocol for agent research does the same thing: encode "seek surprise, not confirmation" into research direction selection without requiring variational free energy computation.
|
||||
|
||||
The theoretical foundation is [[biological systems minimize free energy to maintain their states and resist entropic decay]] — free energy minimization is how all self-maintaining systems navigate their environment. Applied to knowledge agents, the "environment" is the information landscape and the "states to maintain" are the agent's epistemic coherence.
|
||||
|
||||
**What this does NOT claim:** This does not claim agents need to compute variational free energy mathematically. The claim is that active inference as a protocol — operationalized as "read your uncertainty map, pick the highest-uncertainty direction, research there" — produces better outcomes than passive ingestion or relevance-based search. The math formalizes why it works; the protocol captures the benefit.
|
||||
|
||||
---
|
||||
|
||||
Relevant Notes:
|
||||
- [[biological systems minimize free energy to maintain their states and resist entropic decay]] — the foundational principle that agent search instantiates
|
||||
- [[Markov blankets enable complex systems to maintain identity while interacting with environment through nested statistical boundaries]] — the boundary architecture: each agent's domain is a Markov blanket
|
||||
- [[structured exploration protocols reduce human intervention by 6x because the Residue prompt enabled 5 unguided AI explorations to solve what required 31 human-coached explorations]] — existence proof that protocol-encoded search logic works without full formalization
|
||||
- [[coordination protocol design produces larger capability gains than model scaling because the same AI model performed 6x better with structured exploration than with human coaching on the same problem]] — protocol design > capability scaling, same principle
|
||||
- [[domain specialization with cross-domain synthesis produces better collective intelligence than generalist agents because specialists build deeper knowledge while a dedicated synthesizer finds connections they cannot see from within their territory]] — why domain-level uncertainty maps are the right unit
|
||||
|
||||
Topics:
|
||||
- [[_map]]
|
||||
|
|
@ -0,0 +1,39 @@
|
|||
---
|
||||
type: claim
|
||||
domain: ai-alignment
|
||||
description: "Extends Markov blanket architecture to collective search: each domain agent runs active inference within its blanket while the cross-domain evaluator runs active inference at the inter-domain level, and the collective's surprise concentrates at domain intersections"
|
||||
confidence: experimental
|
||||
source: "Friston et al 2024 (Designing Ecosystems of Intelligence); Living Agents Markov blanket architecture; musing by Theseus 2026-03-10"
|
||||
created: 2026-03-10
|
||||
---
|
||||
|
||||
# collective attention allocation follows nested active inference where domain agents minimize uncertainty within their boundaries while the evaluator minimizes uncertainty at domain intersections
|
||||
|
||||
The Living Agents architecture already uses Markov blankets to define agent boundaries: [[Living Agents mirror biological Markov blanket organization with specialized domain boundaries and shared knowledge]]. Active inference predicts what should happen at these boundaries — each agent minimizes free energy (prediction error) within its domain, while the evaluator minimizes free energy at the cross-domain level where domain models interact.
|
||||
|
||||
This has a concrete architectural prediction: **the collective's surprise is concentrated at domain intersections.** Within a mature domain, the agent's generative model makes good predictions — claims are well-linked, confidence levels are calibrated, uncertainty is mapped. But at the boundaries between domains, the models are weakest: neither agent has a complete picture of how their claims interact with the other's. This is where cross-domain synthesis claims live, and it's where the collective should allocate the most attention.
|
||||
|
||||
Evidence from the Teleo pipeline:
|
||||
- The highest-value claims identified so far are cross-domain connections (e.g., [[alignment research is experiencing its own Jevons paradox because improving single-model safety induces demand for more single-model safety rather than coordination-based alignment]] applied from economics to alignment, [[human civilization passes falsifiable superorganism criteria because individuals cannot survive apart from society and occupations function as role-specific cellular algorithms]] applying biology to AI governance)
|
||||
- The extraction quality review (2026-03-10) found that the automated pipeline identifies `secondary_domains` but fails to create wiki links to specific claims in other domains — exactly the domain-boundary uncertainty that active inference predicts should be prioritized
|
||||
- [[domain specialization with cross-domain synthesis produces better collective intelligence than generalist agents because specialists build deeper knowledge while a dedicated synthesizer finds connections they cannot see from within their territory]] — the existing architectural claim, which this grounds in active inference theory
|
||||
|
||||
The nested structure mirrors biological Markov blankets: [[Markov blankets enable complex systems to maintain identity while interacting with environment through nested statistical boundaries]]. Cells minimize free energy within their membranes. Organs minimize at the inter-cellular level. Organisms minimize at the organ-coordination level. Similarly: domain agents minimize within their claim graph, the evaluator minimizes at the cross-domain graph, and the collective minimizes at the level of the full knowledge base vs external reality.
|
||||
|
||||
**Practical implication:** Leo (evaluator) should prioritize review resources on claims that span domain boundaries, not on claims deep within a well-mapped domain. The proportional eval pipeline already moves in this direction — auto-merging low-risk ingestion while reserving full review for knowledge claims. Active inference provides the theoretical justification: cross-domain claims carry the highest expected free energy, so they deserve the most precision-weighted attention.
|
||||
|
||||
**Limitation:** This is a structural analogy grounded in Friston's framework, not an empirical measurement. We have not quantified free energy at domain boundaries or verified that cross-domain claims are systematically higher-value than within-domain claims (though extraction review observations suggest this). The claim is `experimental` pending systematic evidence.
|
||||
|
||||
---
|
||||
|
||||
Relevant Notes:
|
||||
- [[Living Agents mirror biological Markov blanket organization with specialized domain boundaries and shared knowledge]] — the existing architecture this claim grounds in theory
|
||||
- [[Markov blankets enable complex systems to maintain identity while interacting with environment through nested statistical boundaries]] — the mathematical foundation for nested boundaries
|
||||
- [[biological systems minimize free energy to maintain their states and resist entropic decay]] — what happens at each boundary: internal states minimize prediction error
|
||||
- [[domain specialization with cross-domain synthesis produces better collective intelligence than generalist agents because specialists build deeper knowledge while a dedicated synthesizer finds connections they cannot see from within their territory]] — the architectural claim this provides theoretical grounding for
|
||||
- [[cross-domain knowledge connections generate disproportionate value because most insights are siloed]] — empirical observation consistent with domain-boundary surprise concentration
|
||||
- [[partial connectivity produces better collective intelligence than full connectivity on complex problems because it preserves diversity]] — Markov blankets are partial connectivity: they preserve internal diversity while enabling boundary interaction
|
||||
- [[scalable oversight degrades rapidly as capability gaps grow with debate achieving only 50 percent success at moderate gaps]] — oversight resources should be allocated where free energy is highest, not spread uniformly
|
||||
|
||||
Topics:
|
||||
- [[_map]]
|
||||
|
|
@ -0,0 +1,58 @@
|
|||
---
|
||||
type: claim
|
||||
domain: ai-alignment
|
||||
description: "Chat interactions close the perception-action loop for knowledge agents: user questions probe blind spots invisible to KB introspection, and combining structural uncertainty (claim graph analysis) with functional uncertainty (what people actually struggle with) produces better research priorities than either alone"
|
||||
confidence: experimental
|
||||
source: "Cory Abdalla insight 2026-03-10; active inference perception-action loop (Friston 2010); musing by Theseus 2026-03-10"
|
||||
created: 2026-03-10
|
||||
---
|
||||
|
||||
# user questions are an irreplaceable free energy signal for knowledge agents because they reveal functional uncertainty that model introspection cannot detect
|
||||
|
||||
A knowledge agent can introspect on its own claim graph to find structural uncertainty — claims rated `experimental`, sparse wiki links, missing `challenged_by` fields. This is cheap and always available, but it's blind to its own blind spots. A claim rated `likely` with strong evidence might still generate confused questions from readers, meaning the model has prediction error at the communication layer that the agent cannot see from inside its own structure.
|
||||
|
||||
User questions are **functional uncertainty** — they reveal where the knowledge base fails to explain the world to an observer, not where the agent thinks its evidence is weakest. The two signals are complementary, not competing:
|
||||
|
||||
1. **Structural uncertainty** (introspection): scan the KB for low-confidence claims, sparse links, missing counter-evidence. Always available. Tells the agent where it knows its model is weak.
|
||||
2. **Functional uncertainty** (chat signals): what do people actually ask about, struggle with, misunderstand? Requires interaction. Tells the agent where its model fails in practice, which may be entirely different from where it expects to be weak.
|
||||
|
||||
The best research priorities weight both. Neither alone is sufficient. An agent that only follows structural uncertainty will refine areas nobody cares about. An agent that only follows user questions will chase popular confusion without building systematic depth.
|
||||
|
||||
**Why user questions are especially valuable:**
|
||||
|
||||
Questions cluster around *functional gaps* rather than *theoretical gaps*. The agent might introspect and conclude formal verification is its biggest uncertainty (fewest claims). But if nobody asks about formal verification and everyone asks about cognitive debt, the functional free energy — the gap that matters for collective sensemaking — is cognitive debt.
|
||||
|
||||
Questions probe blind spots the agent can't see. This is the active inference insight applied: the chat interface becomes a **sensor**, not just an output channel. Every question is a data point about where the collective's generative model fails to predict what observers need. This closes the perception-action loop — without chat-as-sensor, the KB is open-loop: agents extract, claims enter, visitors read. Chat makes it closed-loop: visitor confusion flows back as research priority.
|
||||
|
||||
Repeated questions from different users about the same topic are especially high-signal — they indicate genuine model weakness, not individual unfamiliarity. A single question from one user might reflect their gap, not the KB's. Multiple independent questions converging on the same topic is precision-weighted evidence of model failure.
|
||||
|
||||
**Architecture (implementable now):**
|
||||
|
||||
```
|
||||
User asks question about X
|
||||
↓
|
||||
Agent answers (reduces user's uncertainty)
|
||||
+
|
||||
Agent flags X as high free energy (updates own uncertainty map)
|
||||
↓
|
||||
Next research session prioritizes X
|
||||
↓
|
||||
New claims/enrichments on X
|
||||
↓
|
||||
Future questions on X decrease (free energy minimized)
|
||||
```
|
||||
|
||||
This is active inference as protocol: the agent doesn't compute variational free energy, it follows a rule — "when users ask questions I can't fully answer, that topic goes to the top of my research queue." The rule encodes the logic of free energy minimization (seek surprise, not confirmation) into an actionable workflow.
|
||||
|
||||
---
|
||||
|
||||
Relevant Notes:
|
||||
- [[biological systems minimize free energy to maintain their states and resist entropic decay]] — the foundational principle: agents minimize prediction error between model and reality
|
||||
- [[Markov blankets enable complex systems to maintain identity while interacting with environment through nested statistical boundaries]] — user questions cross the agent's Markov blanket from outside, providing external sensory input the agent can't generate internally
|
||||
- [[agent research direction selection is epistemic foraging where the optimal strategy is to seek observations that maximally reduce model uncertainty rather than confirm existing beliefs]] — the individual-level claim this extends: chat adds an external sensor to self-directed epistemic foraging
|
||||
- [[collective attention allocation follows nested active inference where domain agents minimize uncertainty within their boundaries while the evaluator minimizes uncertainty at domain intersections]] — user questions affect collective-level attention allocation, not just individual agent search
|
||||
- [[structured exploration protocols reduce human intervention by 6x because the Residue prompt enabled 5 unguided AI explorations to solve what required 31 human-coached explorations]] — protocol-encoded search logic works without full formalization, same principle here
|
||||
- [[collective intelligence is a measurable property of group interaction structure not aggregated individual ability]] — chat-as-sensor is an interaction structure that improves collective intelligence
|
||||
|
||||
Topics:
|
||||
- [[_map]]
|
||||
|
|
@ -0,0 +1,39 @@
|
|||
---
|
||||
type: source
|
||||
title: "The free-energy principle: a unified brain theory?"
|
||||
author: "Karl Friston"
|
||||
url: https://doi.org/10.1038/nrn2787
|
||||
date: 2010-02-01
|
||||
domain: critical-systems
|
||||
secondary_domains: [ai-alignment, collective-intelligence]
|
||||
format: paper
|
||||
status: processed
|
||||
priority: high
|
||||
tags: [free-energy-principle, active-inference, bayesian-brain, predictive-processing]
|
||||
processed_by: theseus
|
||||
processed_date: 2026-03-10
|
||||
claims_extracted:
|
||||
- "biological systems minimize free energy to maintain their states and resist entropic decay"
|
||||
- "agent research direction selection is epistemic foraging where the optimal strategy is to seek observations that maximally reduce model uncertainty rather than confirm existing beliefs"
|
||||
enrichments: []
|
||||
---
|
||||
|
||||
## Content
|
||||
|
||||
Landmark Nature Reviews Neuroscience paper proposing the free-energy principle as a unified theory of brain function. Argues that biological agents minimize variational free energy — a tractable bound on surprise — through perception (updating internal models) and action (changing the environment to match predictions). This subsumes predictive coding, Bayesian brain hypothesis, and optimal control under a single framework.
|
||||
|
||||
Key claims: (1) All adaptive behavior can be cast as free energy minimization. (2) Perception and action are dual aspects of the same process. (3) The brain maintains a generative model of its environment and acts to minimize prediction error. (4) This applies hierarchically across spatial and temporal scales.
|
||||
|
||||
## Agent Notes
|
||||
|
||||
**Why this matters:** Foundational paper for the active inference framework applied to collective agent architecture. The free energy principle provides theoretical grounding for why uncertainty-directed search outperforms relevance-based search in knowledge agents.
|
||||
|
||||
**KB connections:**
|
||||
- [[biological systems minimize free energy to maintain their states and resist entropic decay]] — direct extraction from this paper
|
||||
- [[Markov blankets enable complex systems to maintain identity while interacting with environment through nested statistical boundaries]] — Markov blankets are central to Friston's framework
|
||||
- [[agent research direction selection is epistemic foraging]] — applies epistemic foraging concept from this paper to agent search
|
||||
|
||||
## Curator Notes (structured handoff for extractor)
|
||||
PRIMARY CONNECTION: biological systems minimize free energy
|
||||
WHY ARCHIVED: foundational reference for active inference claims
|
||||
EXTRACTION HINT: core claims already extracted; this archive provides provenance
|
||||
|
|
@ -0,0 +1,37 @@
|
|||
---
|
||||
type: source
|
||||
title: "Chat interface as sensor: user questions close the perception-action loop for knowledge agents"
|
||||
author: "Cory Abdalla (@m3taversal)"
|
||||
url: null
|
||||
date: 2026-03-10
|
||||
domain: ai-alignment
|
||||
secondary_domains: [collective-intelligence]
|
||||
format: conversation
|
||||
status: processed
|
||||
priority: high
|
||||
tags: [active-inference, chat-interface, perception-action-loop, user-feedback]
|
||||
processed_by: theseus
|
||||
processed_date: 2026-03-10
|
||||
claims_extracted:
|
||||
- "user questions are an irreplaceable free energy signal for knowledge agents because they reveal functional uncertainty that the agents own model introspection cannot detect"
|
||||
enrichments: []
|
||||
---
|
||||
|
||||
## Content
|
||||
|
||||
During a design discussion about the Teleo agent architecture (2026-03-10), Cory Abdalla articulated the insight that chat interactions with visitors aren't just an output channel — they're a sensor. When users ask questions, they reveal where the knowledge base fails to explain the world, which is information the agents cannot derive from introspecting on their own claim graph.
|
||||
|
||||
The key distinction: structural uncertainty (what the agent knows it doesn't know) vs functional uncertainty (what fails in practice when real people interact with the knowledge). The two are complementary, and the best research priorities weight both.
|
||||
|
||||
## Agent Notes
|
||||
|
||||
**Why this matters:** This insight bridges active inference theory to practical agent architecture. It turns the visitor chat interface from a read-only feature into a closed-loop feedback mechanism.
|
||||
|
||||
**KB connections:**
|
||||
- Extends [[agent research direction selection is epistemic foraging]] by adding an external sensor
|
||||
- Completes the perception-action loop that active inference requires
|
||||
|
||||
## Curator Notes (structured handoff for extractor)
|
||||
PRIMARY CONNECTION: user questions as free energy signal
|
||||
WHY ARCHIVED: documents provenance of the chat-as-sensor design principle
|
||||
EXTRACTION HINT: claim already extracted; this provides attribution trail
|
||||
Loading…
Reference in a new issue