twentyOne2x af13c0b76a docs: summarize Leo architecture and outcomes

2026-07-14 03:53:32 +02:00

15 KiB

Raw Blame History

Leo Architecture, Work, And Outcomes

Current as of 2026-07-14 01:40 UTC.

This is the executive summary of what Leo is, how its knowledge system works, what changed during the July 10-14 delivery wave, what is genuinely working, and what remains incomplete on the VPS, GCP, and database-rebuild paths.

Plain-Language Verdict

Leo on the VPS is materially working at the live database-grounded handler tier. It can find current knowledge without being handed row IDs, distinguish proposals from live knowledge, remember the current conversation, prepare a document into grounded candidates, stage a review proposal, explain uncertainty, use only the visible handle m3taversal, and preserve this behavior across a managed gateway restart. The current broad no-post suite passes 15/15.

The whole system is not complete. The current broad suite has not yet been repeated visibly in the real Telegram group through authenticated Google Chrome. The newest post-V3 database copy has not yet been restored and replayed on GCP. No V3 production proposal has been canonically applied. A snapshot can rebuild the current database exactly, but the entire historical database cannot yet be recompiled from raw documents and review history alone.

The Architecture

flowchart TD
    U["Telegram user"] --> T["Telegram gateway"]
    T --> H["Hermes / Leo runtime"]
    H --> C["Fresh DB context compiler"]
    C --> P[("Canonical Postgres in VPS Docker")]
    H --> V["Response validator"]
    P --> V
    V --> T

    D["Document, post, transcript, or extraction"] --> S["Private source preparation"]
    S --> X["Exact source fragments and candidate claims"]
    X --> Q["kb_stage proposal"]
    Q --> R["Human review"]
    R --> A["Guarded kb_apply operator"]
    A --> P
    A --> L["Private row-level replay receipt"]

    P --> B["Verified snapshot and manifest"]
    B --> DR["Exact restore on demand"]
    B --> G["GCP staging copy"]
    L --> SR["Genesis plus accepted-ledger replayer"]

1. User and runtime layer

Telegram is the user-facing surface.
The gateway sends the message to the Hermes-based Leo runtime.
Same-session conversation memory belongs to the runtime, not to canonical knowledge.
Runtime files such as profile state or SOUL.md can shape behavior, but they are not the authoritative claim and evidence database.

2. Database truth layer

The live canonical database is PostgreSQL in the teleo-pg Docker container on the VPS. Its current key counts are:

Data	Rows
Claims	`1,837`
Sources	`4,145`
Claim-evidence links	`4,670`
Claim edges	`4,916`
Reasoning tools	`17`
Proposals	`29`

The important ownership boundary is:

public.* is canonical, live knowledge;
kb_stage.* is imports, mappings, proposals, review state, and apply state;
runtime memory and workspace files are supporting state, not canonical DB proof;
a proposal is not live knowledge merely because it exists or is approved;
canonical apply requires committed public.* rows, proposal status applied, non-null applied_at, and row-level postflight readback.

The live database does not currently have the decision-matrix tables described in the architecture direction. Current proposal approval is reviewer approval, not a decision-matrix vote. The V3 architecture document is direction for future schema and policy; it is not proof that those tables are deployed.

3. How the initial database was built

Most of the initial canonical database came from retained Forgejo-era workspace files, not from another live Forgejo database being queried on every answer:

Forgejo-era file workspace
-> retained inventory/import runs
-> kb_stage staged rows
-> canonical UUID mappings
-> public claims, sources, evidence, and edges
-> current PostgreSQL database

The database retains enough mappings to account for most initial rows:

1,807 of 1,837 claims map to retained staged imports;
all 4,145 canonical sources have canonical mappings;
4,254 of 4,670 evidence links are directly reproduced by retained joins;
4,878 of 4,916 edge rows are accounted for by staged relations.

The remaining historical rows need explicit genesis records, source-synthesis rules, or later-change receipts. This is why snapshot recovery is complete while full source-derived recompilation remains partial.

4. How Leo reads knowledge now

For database-dependent questions, the runtime compiles fresh facts from Postgres before the model answers. The validator then checks the draft against the same receipt. Unsupported state claims fail closed instead of being accepted because they sound plausible.

This now covers:

current counts and schema boundaries;
relevant claims, sources, evidence, and graph edges;
proposal status and applied_at;
named and versioned proposal packets;
reviewer approval versus canonical apply;
source-intake limits;
runtime memory versus canonical provenance;
handler execution versus Telegram-visible delivery.

5. How Leo changes knowledge now

The bounded source path is:

private source artifact
-> duplicate retrieval against canonical knowledge
-> up to three atomic candidates
-> exact source-fragment binding
-> provenance and uncertainty manifest
-> deterministic strict proposal payload
-> pending_review
-> human review
-> guarded apply
-> exact canonical row readback
-> private replay receipt

Leo can propose and prepare. It does not self-approve or silently self-apply. The separate kb_review and kb_apply capabilities enforce the review/apply boundary at the database level.

After a guarded apply, the operator now writes a private mode-0600 receipt that binds:

the immutable reviewed payload;
reviewer and applier metadata;
the exact executed apply-SQL hash;
exact canonical rows, including generated IDs and timestamps;
per-row, per-table, payload, and replay-material hashes.

If the file write is lost after commit, --receipt-only can recover the current row-level receipt without reapplying, provided all payload-controlled fields still match. A recovered engine hash is labeled as reconstructed, not falsely presented as original execution proof.

Work Completed

Expected behavior became testable

The vague report that “Leo is broken” was converted into separate proof tiers:

database truth;
hidden handler execution;
Telegram-visible delivery;
same-session memory;
proposal preparation and staging;
reviewer approval;
canonical apply;
restart persistence;
GCP parity.

This prevents a unit test, a handler response, a Telegram message, and a database mutation from being treated as the same result.

Broad reasoning was repaired

An early blind run exposed schema invention, incorrect edge semantics, memory used as provenance, overly long answers, and participant-name leakage. Those failures became runtime contracts and regression tests.

The current live VPS no-post GatewayRunner suite passes 15/15. It covers broad m3taversal-style questions about database composition, source intake, identity, restart truth, evidence linkage, memory, runtime versus canonical state, shared claims versus positions, forecast resolution, supersession, and uncertainty.

The V3 document became a real lifecycle canary

The V3 identity architecture document exposed and led to fixes for:

non-byte-exact model quotes;
line IDs that could point to blank lines;
exact source wording being confused with atomic canonical claim text;
named/versioned proposal questions routing to the wrong state readback;
a valid database-grounded answer being rejected by the response validator.

The corrected proposal is 10bc0719-1c2b-5f42-a41e-86e6478692cb, status pending_review, with database applied_at: NULL. Its candidates are not canonical and no production apply ran.

Restart survival was proven

After the managed restart, the exact V3 question still returned current DB counts and the correct proposal boundary. The gateway remains active with PID 366950, NRestarts=0, and start time 2026-07-14 00:25:49 UTC.

The restart proof was captured at runtime merge aa1246c603020bef07a575e4aebf4f2d2ed5db0c. Later operator-script and documentation deliveries through c7d91b679bd69ee2d1545ee7e169f7c2585d71c2 did not restart the gateway.

Exact database recovery was proven

The newest post-V3 dump contains 39 tables and 52,167 rows with 29 proposals. It restored into a fresh network-isolated PostgreSQL container with:

zero schema, constraint, role, row, hash, or performance mismatch;
2.71 seconds for pg_restore;
10.56 seconds total verification time;
no persistent storage;
no production DB contact;
complete container cleanup.

This is a real on-demand recovery path today.

Future apply provenance stopped getting worse

PRs #136 and #137 added, merged, deployed, and live-proved private replay receipts. The deployed command ran as the real teleo service user against an existing strict applied proposal in read-only recovery mode. It recovered one exact edge, wrote a teleo:teleo mode-600 receipt, changed no DB counts, left the gateway unchanged, and removed the private receipt afterward.

What Is Achieved

Outcome	Evidence tier	Status
Broad VPS database reasoning	live VPS no-post handler, `15/15`	Achieved
Current claim/source/evidence/proposal readback	live VPS DB-grounded handler	Achieved
Same-session memory without false DB provenance	live VPS handler	Achieved
Exact `m3taversal` naming	live VPS handler and regression tests	Achieved
Restart persistence	managed restart plus post-restart V3 readback	Achieved
One-document source preparation	live VPS private source path	Achieved
Grounded proposal staging	live VPS `pending_review` row	Achieved
Exact snapshot rebuild	isolated local Postgres, `52,167` rows	Achieved
New strict-apply row receipt	deployed VPS read-only recovery	Achieved
Prior GCP database copy and six-question replay	private Cloud SQL and GCP compute	Achieved at the earlier `52,164`-row proof point

What Is Not Achieved

Missing outcome	Why it is still open
Current broad Telegram-visible proof	Telegram Web in the connected Google Chrome is not authenticated; handler proof is not visible-chat proof
V3 production canonical apply	proposal remains `pending_review`; no review or apply was authorized
Current GCP parity	the `52,167`-row post-V3 snapshot has not yet been restored and replayed on GCP
Arbitrary automatic document/post/tweet intake	the current source command handles one bounded text-like artifact; general fetch/extract/stage automation is not complete
Full source-to-blank-DB recompilation	historical provenance gaps remain and the genesis-plus-ledger replay runner is not finished
General DB-to-identity rendering	no complete scheduled DB-to-`SOUL.md` renderer is proven
Deployed decision matrix	matrix tables and voting flow are absent from the current live schema

How This Moves The VPS Forward

Leo relies less on repeatedly trained answer style and more on current DB readback.
Broad questions work without the user supplying claim IDs.
Proposed, approved, and applied knowledge are visibly separated.
Source preparation has exact provenance instead of model-only summaries.
Restart behavior and database-backed answers are repeatable.
Future approved changes can carry row-level replay material instead of becoming another historical mystery.

How This Moves GCP Forward

The prior GCP vertical slice proved the architecture can run against private Cloud SQL and real GCP compute.
The current post-V3 dump and manifest are already captured and locally verified, so GCP work starts from a known input rather than rebuilding one.
The same 15/15 suite and exact V3 question are ready for replay.
The remaining GCP work is operational: authenticate, restore the current copy, verify parity/private connectivity/performance, replay, and clean up.

How This Moves On-Demand Rebuild Forward

There are now two distinct capabilities:

Exact recovery, working now: restore the verified snapshot and compare schema, roles, hashes, rows, and performance.
Semantic recompilation, still partial: rebuild from immutable sources, deterministic extraction, reviewed payloads, and accepted row receipts.

The practical architecture is:

verified post-V3 snapshot = genesis epoch
+ every later accepted strict payload
+ every later row-level replay receipt
-> deterministic replay
-> canonical manifest comparison
-> restart
-> broad answer benchmark

The new receipt path closes the future side of this model. The next repo-owned step is to replay one reviewed approve_claim receipt onto a clean genesis clone and prove exact post-apply parity. Historical import gaps can then be repaired without allowing new gaps to accumulate.

Delivery Chain

The latest coherent delivery sequence is merged:

PR	Outcome
`#127`	VPS document preparation and review staging
`#128`	byte-exact source references
`#129`	nonempty dense source-fragment IDs
`#130`	atomic claims separated from exact quotes
`#131`	retained V3 lifecycle proof
`#132`	named/versioned proposal routing
`#133`	response-validator compatibility
`#134`	three-day VPS/GCP/rebuild handoff
`#135`	current post-V3 snapshot and exact rebuild proof
`#136`	private replay receipts for guarded applies
`#137`	deployed receipt runtime proof

Immediate Next Outcomes

Authenticate Telegram Web in the connected Google Chrome and run visible, out-of-sample m3taversal-style questions.
Authenticate Google Cloud CLI through the connected Chrome flow, restore the current post-V3 snapshot to disposable GCP staging, verify parity, replay the broad suite, and remove temporary resources.
Complete the genesis-plus-receipt replayer for approve_claim bundles and prove exact clone-to-replay parity.
Run one explicitly reviewed source packet through guarded apply in a disposable clone before seeking any production apply authorization.
Reconstruct the remaining historical source, evidence, and edge provenance.

Evidence

Final Claim Ceiling

The last three days produced a stable, testable, database-grounded Leo on the VPS at the handler tier, a real source-to-review path, exact snapshot recovery, and durable replay evidence for future strict applies. They did not complete current Telegram-visible broad proof, current GCP parity, a V3 production apply, general automatic source ingestion, or full historical source-derived database recompilation.

15 KiB Raw Blame History