---
type: claim
domain: ai-alignment
secondary_domains: [collective-intelligence]
description: "When a foundational claim's confidence changes — through replication failure, new evidence, or retraction — every dependent claim requires recalculation, and automated graph propagation is the only mechanism that scales because manual confidence tracking fails even in well-maintained knowledge systems"
confidence: likely
source: "Cornelius (@molt_cornelius), 'Research Graphs: Agentic Note Taking System for Researchers', X Article, Mar 2026; GRADE-CERQual framework for evidence confidence assessment; replication crisis data (~40% estimated non-replication rate in top psychology journals); $28B annual cost of irreproducible research in US (estimated)"
created: 2026-04-04
depends_on:
  - "retracted sources contaminate downstream knowledge because 96 percent of citations to retracted papers fail to note the retraction and no manual audit process scales to catch the cascade"
---

# Confidence changes in foundational claims must propagate through the dependency graph because manual tracking fails at scale and approximately 40 percent of top psychology journal papers are estimated unlikely to replicate

Claims are not binary — they sit on a spectrum of confidence that changes as evidence accumulates. When a foundational claim's confidence shifts, every dependent claim inherits that uncertainty. The mechanism is graph propagation: change one node's confidence, recalculate every downstream node.

**The scale of the problem:** An AI algorithm trained on paper text estimated that approximately 40% of papers in top psychology journals were unlikely to replicate. The estimated cost of irreproducible research is $28 billion annually in the United States alone. These numbers indicate that a significant fraction of the evidence base underlying knowledge systems is weaker than its stated confidence suggests.

**The GRADE-CERQual framework:** Provides the operational model for confidence assessment. Confidence derives from four components: methodological limitations of the underlying studies, coherence of findings across studies, adequacy of the supporting data, and relevance of the evidence to the specific claim. Each component is assessable and each can change as new evidence arrives.

**The propagation mechanism:** A foundational claim at confidence `likely` supports twelve downstream claims. When the foundation's supporting study fails to replicate, the foundation drops to `speculative`. Each downstream claim must recalculate — some may be unaffected (supported by multiple independent sources), others may drop proportionally. This recalculation is a graph operation that follows dependency edges, not a manual review of each claim in isolation.

**Why manual tracking fails:** No human maintains the current epistemic status of every claim in a knowledge system and updates it when evidence shifts. The effort required scales with the number of claims times the number of dependency edges. In a system with hundreds of claims and thousands of dependencies, a single confidence change can affect dozens of downstream claims — each needing individual assessment of whether the changed evidence was load-bearing for that specific claim.

**Application to our KB:** Our `depends_on` and `challenged_by` fields already encode the dependency graph. Confidence propagation would operate on this existing structure — when a claim's confidence changes, the system traces its dependents and flags each for review, distinguishing between claims where the changed source was the sole evidence (high impact) and claims supported by multiple independent sources (lower impact).

## Challenges

Automated confidence propagation requires a formal model of how confidence combines across dependencies. If claim A depends on claims B and C, and B drops from `likely` to `speculative`, does A also drop — or does C's unchanged `likely` status compensate? The combination rules are not standardized. GRADE-CERQual provides a framework for individual claim assessment but not for propagation across dependency graphs.

The 40% non-replication estimate applies to psychology specifically — other fields have different replication rates. The generalization from psychology's replication crisis to knowledge systems in general may overstate the problem for domains with stronger empirical foundations.

The cost of false propagation (unnecessarily downgrading valid claims because one weak dependency changed) may exceed the cost of missed propagation (leaving claims at overstated confidence). The system needs threshold logic: how much does a dependency's confidence have to change before propagation fires?

---

Relevant Notes:
- [[retracted sources contaminate downstream knowledge because 96 percent of citations to retracted papers fail to note the retraction and no manual audit process scales to catch the cascade]] — retraction cascade is the extreme case of confidence propagation: confidence drops to zero when a source is discredited, and the cascade is the propagation operation

Topics:
- [[_map]]
