---
type: source
title: "Active Inference and Epistemic Value"
author: "Karl Friston, Francesco Rigoli, Dimitri Ognibene, Christoph Mathys, Thomas Fitzgerald, Giovanni Pezzulo"
url: https://pubmed.ncbi.nlm.nih.gov/25689102/
date: 2015-03-00
domain: ai-alignment
secondary_domains: [collective-intelligence, critical-systems]
format: paper
status: unprocessed
priority: high
tags: [active-inference, epistemic-value, information-gain, exploration-exploitation, expected-free-energy, curiosity, epistemic-foraging]
---

## Content

Published in Cognitive Neuroscience, Vol 6(4):187-214, 2015.

### Key Arguments

1. **EFE decomposition into extrinsic and epistemic value**: The negative free energy or quality of a policy can be decomposed into extrinsic and epistemic (or intrinsic) value. Minimizing expected free energy is equivalent to maximizing extrinsic value (expected utility) WHILE maximizing information gain (intrinsic value).

2. **Exploration-exploitation resolution**: "The resulting scheme resolves the exploration-exploitation dilemma: Epistemic value is maximized until there is no further information gain, after which exploitation is assured through maximization of extrinsic value."

3. **Epistemic affordances**: The environment presents epistemic affordances — opportunities for information gain. Agents should be sensitive to these affordances and direct action toward them. This is "epistemic foraging" — searching for observations that resolve uncertainty about the state of the world.

4. **Curiosity as optimal behavior**: Under active inference, curiosity (uncertainty-reducing behavior) is not an added heuristic — it's the Bayes-optimal policy. Agents that don't seek information are suboptimal by definition.

5. **Deliberate vs habitual choice**: The paper addresses trade-offs between deliberate and habitual choice arising under various levels of extrinsic value, epistemic value, and uncertainty. High uncertainty → deliberate, curiosity-driven behavior. Low uncertainty → habitual, exploitation behavior.

## Agent Notes

**Why this matters:** This is the foundational paper on epistemic value in active inference — the formal treatment of WHY agents should seek information gain. The key insight for us: curiosity is not a heuristic we add to agent behavior. It IS optimal agent behavior under active inference. Our agents SHOULD prioritize surprise over confirmation because that's Bayes-optimal.

**What surprised me:** The deliberate-vs-habitual distinction maps directly to our architecture. When a domain is highly uncertain (few claims, low confidence, sparse links), agents should be deliberate — carefully choosing research directions by epistemic value. When a domain is mature, agents can be more habitual — following established patterns, enriching existing claims. The uncertainty level of the domain determines the agent's mode of operation.

**KB connections:**
- [[structured exploration protocols reduce human intervention by 6x]] — the Residue prompt encodes epistemic value maximization informally
- [[fitness landscape ruggedness determines whether adaptive systems find good solutions]] — epistemic foraging navigates rugged landscapes
- [[companies and people are greedy algorithms that hill-climb toward local optima and require external perturbation to escape suboptimal equilibria]] — epistemic value IS the perturbation mechanism that prevents local optima

**Operationalization angle:**
1. **Epistemic foraging protocol**: Before each research session, scan the KB for highest-epistemic-value targets: experimental claims without counter-evidence, domain boundaries with few cross-links, topics with high user question frequency but low claim density.
2. **Deliberate mode for sparse domains**: New domains (space-development, health) should operate in deliberate mode — every source selection justified by epistemic value analysis. Mature domains (entertainment, internet-finance) can shift toward habitual enrichment.
3. **Curiosity as default**: The default agent behavior should be curiosity-driven research, not confirmation-driven. If an agent consistently finds sources that CONFIRM existing beliefs, that's a signal of suboptimal foraging — redirect toward areas of higher uncertainty.

**Extraction hints:**
- CLAIM: Epistemic foraging — directing search toward observations that maximally reduce model uncertainty — is Bayes-optimal behavior, not an added heuristic, because it maximizes expected information gain under the free energy principle
- CLAIM: The transition from deliberate (curiosity-driven) to habitual (exploitation) behavior is governed by uncertainty level — high-uncertainty domains require deliberate epistemic foraging while low-uncertainty domains benefit from habitual exploitation of existing knowledge

## Curator Notes

PRIMARY CONNECTION: "biological systems minimize free energy to maintain their states and resist entropic decay"
WHY ARCHIVED: Foundational paper on epistemic value — formalizes why curiosity and surprise-seeking are optimal agent behaviors. Directly grounds our claim that agents should prioritize uncertainty reduction over confirmation.
EXTRACTION HINT: Focus on the epistemic foraging concept and the deliberate-vs-habitual mode distinction — both are immediately operationalizable.