c7b3093fe1
- Source: inbox/archive/2021-03-00-sajid-active-inference-demystified-compared.md - Domain: ai-alignment - Extracted by: headless extraction cron Pentagon-Agent: Theseus <HEADLESS>
2.4 KiB
2.4 KiB
| type | domain | description | confidence | source | created | depends_on | challenged_by | |
|---|---|---|---|---|---|---|---|---|
| claim | ai-alignment | Research direction scoring should weight both epistemic value (claim uncertainty and KB sparsity) and pragmatic value (mission alignment and user relevance), with epistemic value dominating when the KB is sparse | experimental | Sajid, Parr, Ball, and Friston (2021) - Active Inference: Demystified and Compared, Neural Computation Vol 33(3):674-712; operationalization notes from source curator | 2026-03-10 |
|
Research direction scoring requires both epistemic and pragmatic value components
Applying the EFE decomposition to research direction selection provides a principled scoring mechanism. Epistemic value can be operationalized as: how many experimental/speculative claims does this topic have? How sparse are the wiki links? Pragmatic value can be operationalized as: how relevant is this to current objectives and user questions? An agent should research topics that score high on both dimensions, but epistemic value should dominate when the KB is sparse. This prevents the agent from purely exploiting user-aligned topics while leaving entire domains unexplored, ensuring balanced knowledge base development.
Evidence
- 2021-03-00-sajid-active-inference-demystified-compared — Agent notes: "The EFE decomposition is the key to operationalizing active inference for our agents. Epistemic value = 'how much would researching this topic reduce our KB uncertainty?' Pragmatic value = 'how much does this align with our mission objectives?' An agent should research topics that score high on BOTH — but epistemic value should dominate when the KB is sparse."
- 2021-03-00-sajid-active-inference-demystified-compared — Agent notes: "Surprise-weighted extraction: When extracting claims, weight contradictions to existing beliefs HIGHER than confirmations — they have higher epistemic value."
Challenges
[None identified in current literature]
Relevant Notes:
- coordination-protocol-design-produces-larger-capability-gains-than-model-scaling — The scoring mechanism could be viewed as a coordination protocol between exploration and exploitation priorities
Topics: