53 lines
No EOL
5 KiB
Markdown
53 lines
No EOL
5 KiB
Markdown
---
|
|
type: claim
|
|
domain: ai-alignment
|
|
secondary_domains: [collective-intelligence]
|
|
description: "Karpathy's three-layer LLM wiki architecture (raw sources → LLM-compiled wiki → schema) demonstrates that persistent synthesis outperforms retrieval-augmented generation by making cross-references and integration a one-time compile step rather than a per-query cost"
|
|
confidence: experimental
|
|
source: "Andrej Karpathy, 'LLM Knowledge Base' GitHub gist (April 2026, 47K likes, 14.5M views); Mintlify ChromaFS production data (30K+ conversations/day)"
|
|
created: 2026-04-05
|
|
depends_on:
|
|
- one agent one chat is the right default for knowledge contribution because the scaffolding handles complexity not the user
|
|
related:
|
|
- agent native retrieval converges on filesystem abstractions over embedding search because grep cat ls and find are all an agent needs to navigate structured knowledge
|
|
reweave_edges:
|
|
- agent native retrieval converges on filesystem abstractions over embedding search because grep cat ls and find are all an agent needs to navigate structured knowledge|related|2026-04-17
|
|
---
|
|
|
|
# LLM-maintained knowledge bases that compile rather than retrieve represent a paradigm shift from RAG to persistent synthesis because the wiki is a compounding artifact not a query cache
|
|
|
|
Karpathy's LLM Wiki methodology (April 2026) proposes a three-layer architecture that inverts the standard RAG pattern:
|
|
|
|
1. **Raw Sources (immutable)** — curated articles, papers, data files. The LLM reads but never modifies.
|
|
2. **The Wiki (LLM-owned)** — markdown files containing summaries, entity pages, concept pages, interconnected knowledge. "The LLM owns this layer entirely. It creates pages, updates them when new sources arrive, maintains cross-references, and keeps everything consistent."
|
|
3. **The Schema (configuration)** — a specification document (e.g., CLAUDE.md) defining wiki structure, conventions, and workflows. Transforms the LLM from generic chatbot into systematic maintainer.
|
|
|
|
The fundamental difference from RAG: "the LLM doesn't just index it for later retrieval. It reads it, extracts the key information, and integrates it into the existing wiki." Each new source touches 10-15 pages through updates and cross-references, rather than being isolated as embedding chunks for retrieval.
|
|
|
|
## Why compilation beats retrieval
|
|
|
|
RAG treats knowledge as a retrieval problem — store chunks, embed them, return top-K matches per query. This fails when:
|
|
- Answers span multiple documents (no single chunk contains the full answer)
|
|
- The query requires synthesis across domains (embedding similarity doesn't capture structural relationships)
|
|
- Knowledge evolves and earlier chunks become stale without downstream updates
|
|
|
|
Compilation treats knowledge as a maintenance problem — each new source triggers updates across the entire wiki, keeping cross-references current and contradictions surfaced. The tedious work (updating cross-references, tracking contradictions, keeping summaries current) falls to the LLM, which "doesn't get bored, doesn't forget to update a cross-reference, and can touch 15 files in one pass."
|
|
|
|
## The Teleo Codex as existence proof
|
|
|
|
The Teleo collective's knowledge base is a production implementation of this pattern, predating Karpathy's articulation by months. The architecture matches almost exactly: raw sources (inbox/archive/) → LLM-compiled claims with wiki links and frontmatter → schema (CLAUDE.md, schemas/). The key difference: Teleo distributes the compilation across 6 specialized agents with domain boundaries, while Karpathy's version assumes a single LLM maintainer.
|
|
|
|
The 47K-like, 14.5M-view reception suggests the pattern is reaching mainstream AI practitioner awareness. The shift from "building a better RAG pipeline" to "building a better wiki maintainer" has significant implications for knowledge management tooling.
|
|
|
|
## Challenges
|
|
|
|
The compilation model assumes the LLM can reliably synthesize and maintain consistency across hundreds of files. At scale, this introduces accumulating error risk — one bad synthesis propagates through cross-references. Karpathy addresses this with a "lint" operation (health-check for contradictions, stale claims, orphan pages), but the human remains "the editor-in-chief" for verification. The pattern works when the human can spot-check; it may fail when the wiki outgrows human review capacity.
|
|
|
|
---
|
|
|
|
Relevant Notes:
|
|
- [[one agent one chat is the right default for knowledge contribution because the scaffolding handles complexity not the user]] — the Teleo implementation of this pattern: one agent handles all schema complexity, compiling knowledge from conversation into structured claims
|
|
- [[multi-agent coordination delivers value only when three conditions hold simultaneously natural parallelism context overflow and adversarial verification value]] — the Teleo multi-agent version of the wiki pattern meets all three conditions: domain parallelism, context overflow across 400+ claims, adversarial verification via Leo's cross-domain review
|
|
|
|
Topics:
|
|
- [[_map]] |