teleo-infrastructure/docs/kb-rebuild-and-recompile.md

Rebuilding Leo's Knowledge Database

Outcome

Leo should improve by compiling durable source material and reviewed changes into Postgres. Repeatedly changing prompts or retraining the chat behavior is not the knowledge system.

There are two different rebuilds:

1. Exact recovery restores the current canonical database from a verified snapshot. This is working now.
2. Semantic recompilation starts from the retained source corpus and the reviewed change ledger, then reproduces the canonical rows. This is partly recoverable but is not yet complete.

Exact Recovery: Working

Run:

.venv/bin/python ops/run_local_canonical_postgres_rebuild.py \
  --dump /private/path/teleo-canonical.dump \
  --source-manifest /private/path/source-manifest.jsonl \
  --output /tmp/teleo-canonical-rebuild-receipt.json

The retained 2026-07-14 post-V3 canary restored a fresh, network-isolated Postgres and then removed it. The restored target matched the source across all 39 manifest tables and all 52,167 rows, with no schema, data, constraint, role, or performance mismatch. The same source snapshot was subsequently restored to a disposable private-TLS GCP Cloud SQL clone with exact parity, a bounded no-send reasoning turn, and verified cleanup. Key rows included:

• 1,837 claims;
• 4,145 sources;
• 4,670 claim-evidence links;
• 4,916 claim edges;
• 17 reasoning tools;
• 29 proposals.

This snapshot includes the completed V3 source canary. Disposable GCP restore parity is proven at that retained point. Persistent GCP teleo_canonical remains the older staging copy, so ongoing parity, promotion, and production cutover remain unproven.

This is the fastest disaster-recovery path. It does not require Leo to re-extract or relearn the corpus.

Source Recompilation: Current Evidence

Read-only VPS inspection found two retained June import runs. Both point to the Forgejo-era workspace at /opt/teleo-eval/workspaces/main and retained inventory JSONL files under /opt/teleo-eval/kb-import/.

The database still retains:

• kb_stage.import_runs for the two inventories;
• staged claims, sources, claim-source links, and claim edges;
• kb_stage.canonical_mappings from legacy keys to canonical UUIDs;
• mappings for 1,807 of 1,837 canonical claims;
• mappings for all 4,145 canonical sources.

For the 1,807 mapped claims, current canonical type, text, status, confidence, tags, and creator match the retained staged rows exactly. Creation timestamps also follow a recoverable rule: use the legacy timestamp when present and the mapping timestamp for the eight rows that had no legacy timestamp.

The simple retained-row joins currently reproduce:

• 4,254 of 4,670 canonical evidence links;
• 4,878 of 4,916 canonical edge rows can be accounted for by a staged relation; historical duplicate multiplicity still needs an explicit replay rule.

Re-run this aggregate, read-only coverage audit against any restored local clone:

.venv/bin/python ops/audit_kb_rebuild_coverage.py \
  --container <restored-local-postgres-container> \
  --database teleo

The auditor emits no claim bodies or source excerpts and separates snapshot recovery from source-derived recompilation readiness.

The remaining gaps are concrete rather than mysterious:

• 30 claims were created after or outside the mapped import;
• 284 mapped source rows do not have a matching retained staged_sources row and need the original source-synthesis rule or an explicit genesis record;
• 416 evidence links need source-synthesis or later-change provenance;
• 38 edge rows need later-change receipts or explicit replay records;
• old applied proposal rows do not describe every historical canonical write.

This proves that most of the initial database came from the retained file-KB import path. It does not yet prove a clean blank-database recompile.

Target Compiler

The durable rebuild model is:

immutable source corpus + file hashes
-> deterministic inventory and classification
-> staged claims, sources, evidence links, and edges
-> stable canonical ID mapping
-> review decisions
-> append-only accepted apply payloads and receipts
-> canonical Postgres
-> render/sync/restart
-> answer benchmark

Use the current verified snapshot as genesis epoch 1. Preserve its dump, manifest, source commit, inventory files, mappings, and aggregate rebuild receipt. Every accepted change after that epoch must carry a replayable strict apply payload and row-level postflight receipt. This prevents the historical gap from growing while the old import rules are reconstructed.

The guarded apply CLI now enforces the receipt half of that policy. After a successful apply it reloads the immutable applied proposal, selects the exact canonical rows described by the strict payload, binds generated row IDs and timestamps, hashes the exact executed apply SQL, payload, and rows, and atomically writes a private mode 0600 receipt. It also supports read-only recovery for a committed apply whose receipt file was lost, provided every payload-controlled row still matches the immutable reviewed payload:

python3 scripts/apply_proposal.py <proposal-uuid> --receipt-only

The live VPS recovery canary used strict applied proposal 00957f6c-9883-4015-95a4-6b09367efb0e. It recovered exactly one canonical edge, kept all database counts and the proposal payload hash unchanged, left the Leo gateway on the same PID with zero restarts, and removed the temporary private receipt. The full receipt is deliberately not committed because it can contain claim bodies or source excerpts; the sanitized proof is retained as docs/reports/leo-working-state-20260709/kb-apply-replay-receipt-current.json.

Normal applies mark the SQL hash as exact_executed_sql. A later --receipt-only recovery marks it as reconstructed_current_engine; it never pretends the current engine hash is historical proof of the originally executed program.

This closes replay-receipt loss for new strict applies. The receipt alone does not retain every column of the proposal ledger, so exact reconstruction also needs the full approved proposal row, immutable approval snapshot, and final applied proposal row.

Genesis Plus Strict Ledger: Working Deterministic Slice

ops/run_local_genesis_ledger_rebuild.py now executes the first exact genesis-plus-ledger slice in one command:

.venv/bin/python ops/run_local_genesis_ledger_rebuild.py \
  --genesis-dump /private/path/genesis.dump \
  --genesis-manifest /private/path/genesis-manifest.jsonl \
  --ledger /private/path/ledger.json \
  --ledger-sha256 "$LEDGER_SHA256" \
  --output /tmp/genesis-ledger-rebuild-receipt.json

The v1 ledger pins the genesis dump and manifest, final parity manifest, reconstruction/restore/guard/apply/replay/parity engines, and every ordered private material file. Each material file contains one existing kb_apply_replay_receipt, the exact full proposal row immediately before apply, its immutable kb_proposal_approvals row, and the exact full proposal row after apply. These files can contain claim text or source excerpts and must remain private.

The ledger shape is:

{
  "artifact": "teleo_genesis_plus_ledger",
  "contract_version": 1,
  "engine": {
    "reconstruction_command_sha256": "<sha256>",
    "base_rebuild_engine_sha256": "<sha256>",
    "apply_engine_sha256": "<sha256>",
    "replay_receipt_engine_sha256": "<sha256>",
    "guard_prerequisites_sha256": "<sha256>",
    "parity_sql_sha256": "<sha256>"
  },
  "genesis": {
    "dump_sha256": "<sha256>",
    "parity_manifest_sha256": "<sha256>"
  },
  "entries": [{
    "sequence": 1,
    "material": "private/0001.json",
    "sha256": "<material-file-sha256>",
    "replay_material_sha256": "<receipt-replay-material-sha256>"
  }],
  "final_parity": {
    "manifest": "final-manifest.jsonl",
    "sha256": "<sha256>"
  }
}

Each referenced material object has exact top-level keys artifact, contract_version, sequence, approved_proposal, approval_snapshot, applied_proposal, and replay_receipt. Proposal objects must contain every current kb_stage.kb_proposals column; partial envelopes are rejected.

The command verifies every hash before starting Docker, requires a SHA-256-pinned Postgres image (and defaults to a pinned PostgreSQL 16 Alpine multi-platform digest), restores it on tmpfs with --network none, reapplies the current guarded prerequisites, and proves genesis parity. Insert-only entries seed the receipt's exact canonical row IDs and timestamps before the existing kb_apply payload-bound guard executes. For revise_strategy, the runner seeds only the proposal and approval, captures the target agent's prestate, executes the real guarded transition, validates the generated delta, and then replaces only those generated rows with the receipt-pinned IDs and timestamps. Every path checks exact proposal and canonical row readbacks before verifying the complete final parity manifest. Its public mode-0600 receipt contains hashes, IDs, types, counts, parity summaries, and cleanup proof, but no payloads, rows, SQL, source paths, or command errors. The legacy seed_exact summary is the insert-only aggregate of proposal_seed_exact and canonical_seed_exact; it is intentionally false for successful mutating entries, which instead report mutating_delta_validated and mutating_poststate_normalized. Fresh guard bootstrap rows use a fixed baseline timestamp rather than wall clock time, so repeated clean restores have identical row hashes.

The exact v1 claim ceiling is intentionally bounded:

• add_edge, attach_evidence, approve_claim, and revise_strategy strict receipts execute;
• sequence gaps, hash drift, engine drift, duplicate proposals, and legacy or freeform payloads fail before container start;
• revise_strategy is accepted only when the receipt pins the exact SQL that matches the current guarded apply engine. Immediately before each entry, the runner captures the target agent's strategy/node IDs, active strategy, non-retired nodes, and maximum version. It then validates the generated post-minus-pre delta, requires version = previous maximum + 1, and replaces only the generated strategy/node rows with the exact receipt rows;
• the original transaction timestamp is derived from the fresh strategy created_at and must equal every fresh node's created_at and updated_at. It must also fall within the immutable, timezone-aware interval reviewed_at <= transaction timestamp <= applied_at. Only node IDs observed as non-retired before apply receive that timestamp; already-retired, unrelated-agent, and shared NULL-agent rows stay untouched;
• generated nodes must have no anchors before normalization, preventing a delete-and-reinsert step from silently cascading future trigger-created rows;
• full proposal before/after rows are mandatory because the current receipt envelope does not retain proposal origin fields or exact updated_at;
• this proves only an isolated local reconstruction. It does not touch or prove VPS, GCP, Telegram, a live database, or blank-schema source recompilation.

The transition contract for revise_strategy is final-state deterministic, not a claim that the v1 receipt independently contains a historical before-image:

exact genesis/pre-entry state
  + exact current/original guarded SQL
  + receipt-pinned fresh strategy and nodes
  -> prior active strategy inactive
  -> exactly the prior non-retired nodes retired at the original transaction time
  -> one receipt-exact active strategy and receipt-exact node set

The genesis and final manifests remain mandatory oracles. Any incorrect prestate, unrelated-row mutation, missing/extra generated row, semantic drift, version drift, hash drift, or final rowset difference fails the reconstruction.

The source compiler now turns one raw artifact, its strict UTF-8 extraction, and a reviewed extraction manifest into a deterministic, hash-bound pending_review proposal bundle:

.venv/bin/python scripts/compile_kb_source_packet.py \
  --artifact fixtures/working-leo/document-ingestion-v1.json \
  --text fixtures/working-leo/document-ingestion-v1.json \
  --manifest fixtures/working-leo/source-compiler-manifest-v1.json \
  --output /tmp/working-leo-source-packet-v1.json

The compiler verifies artifact and extraction hashes, stable source identity, current schema taxonomies, unique logical keys, and exact claim/evidence quotes. It reuses the existing proposal normalizer and staging preflight, but it has no database connection and executes neither staging nor apply. Its output is the review packet, not canonical knowledge.

The VPS also exposes a bounded preparation command for one text-like filesystem document (or a binary artifact with a separately supplied strict UTF-8 extraction):

teleo-kb prepare-source \
  --artifact /home/teleo/.hermes/profiles/leoclean/state/kb-source-inbox/source.md \
  --identity document:stable-source-id \
  --source-key stable_source_key \
  --source-type article \
  --title "Stable source title" \
  --locator artifact://stable/source-id \
  --output-dir /home/teleo/.hermes/profiles/leoclean/state/kb-source-preparation/source-id

It queries canonical claims before extraction, caps a single document at three new candidates with confidence at or below 0.75, has the model select densely numbered non-empty source fragments, resolves those IDs to exact source substrings, records duplicate judgments, and validates a v2 manifest through the source compiler. Unknown line IDs are rejected rather than fuzzily matched. It writes private files only. The extracted atomic proposition is the proposed claim text; exact source wording remains separately hash-bound as quote and evidence. A separate teleo-kb propose-source call is required to create a pending_review row; neither command applies canonical rows.

The existing full-data clone canary separately proves that a reviewed packet can create source, claim, evidence, and edge rows and affect later reasoning. The remaining reconstruction work is to backfill or explicitly reject legacy freeform applies and extend beyond genesis recovery to a blank-schema source compiler. The strict ledger runner does not prove that every historical canonical row can be rebuilt semantically from retained sources.

Definition Of Working

Semantic recompilation is complete only when all of these pass:

1. A command creates a blank database from the retained source corpus plus the reviewed ledger.
2. Schema, constraints, roles, table counts, row hashes, and key query results match the canonical manifest.
3. Every canonical row traces to a genesis import record or a reviewed apply receipt.
4. A new document can be hash-captured, extracted into grounded candidates, deduplicated, staged, reviewed, applied in a disposable clone, and read back.
5. After render/sync and restart, Leo answers the related broad question using the new rows and cites the source chain.

Until then, exact snapshot recovery is production recovery; source recompilation is an active build capability, not a finished claim.