teleo-infrastructure/docs/reports/leo-working-state-20260709/leo-architecture-work-and-outcomes-20260714.md
twentyOne2x 15c30f1fbe
Some checks are pending
CI / lint-and-test (push) Waiting to run
Make Leo skill onboarding clean-context reproducible (#150)
* make Leo skill onboarding clean-context reproducible

* harden Leo clean-context skill acceptance
2026-07-15 03:19:15 +02:00

16 KiB

Leo Architecture, Work, And Outcomes

Original delivery summary from 2026-07-14 01:40 UTC, reconciled with merged repository truth on 2026-07-15.

This is the executive summary of what Leo is, how its knowledge system works, what changed during the July 10-14 delivery wave, what is genuinely working, and what remains incomplete on the VPS, GCP, and database-rebuild paths.

Plain-Language Verdict

Leo on the VPS is materially working at the live database-grounded handler tier. It can find current knowledge without being handed row IDs, distinguish proposals from live knowledge, remember the current conversation, prepare a document into grounded candidates, stage a review proposal, explain uncertainty, use only the visible handle m3taversal, and preserve this behavior across a managed gateway restart. The current broad no-post suite passes 15/15.

The whole system is not complete. The current broad suite has not yet been repeated visibly in the real Telegram group through authenticated Google Chrome. The newest post-V3 database copy has since been restored to a disposable private GCP Cloud SQL clone with exact parity and a bounded no-send reasoning replay, then cleaned up. Persistent GCP remains staging and older than that disposable proof point. No V3 production proposal has been canonically applied. A snapshot can rebuild the current database exactly, and merged PR #146 adds an exact insert-only genesis-plus-ledger slice; the entire historical database still cannot be recompiled from raw documents and review history alone.

Repository reconciliation: PRs #146 and #147 are merged main truth. PR #148 remains an open least-privilege GCP candidate and is not canonical runtime proof.

The Architecture

flowchart TD
    U["Telegram user"] --> T["Telegram gateway"]
    T --> H["Hermes / Leo runtime"]
    H --> C["Fresh DB context compiler"]
    C --> P[("Canonical Postgres in VPS Docker")]
    H --> V["Response validator"]
    P --> V
    V --> T

    D["Document, post, transcript, or extraction"] --> S["Private source preparation"]
    S --> X["Exact source fragments and candidate claims"]
    X --> Q["kb_stage proposal"]
    Q --> R["Human review"]
    R --> A["Guarded kb_apply operator"]
    A --> P
    A --> L["Private row-level replay receipt"]

    P --> B["Verified snapshot and manifest"]
    B --> DR["Exact restore on demand"]
    B --> G["GCP staging copy"]
    L --> SR["Genesis plus accepted-ledger replayer"]

1. User and runtime layer

  • Telegram is the user-facing surface.
  • The gateway sends the message to the Hermes-based Leo runtime.
  • Same-session conversation memory belongs to the runtime, not to canonical knowledge.
  • Runtime files such as profile state or SOUL.md can shape behavior, but they are not the authoritative claim and evidence database.

2. Database truth layer

The live canonical database is PostgreSQL in the teleo-pg Docker container on the VPS. Its current key counts are:

Data Rows
Claims 1,837
Sources 4,145
Claim-evidence links 4,670
Claim edges 4,916
Reasoning tools 17
Proposals 29

The important ownership boundary is:

  • public.* is canonical, live knowledge;
  • kb_stage.* is imports, mappings, proposals, review state, and apply state;
  • runtime memory and workspace files are supporting state, not canonical DB proof;
  • a proposal is not live knowledge merely because it exists or is approved;
  • canonical apply requires committed public.* rows, proposal status applied, non-null applied_at, and row-level postflight readback.

The live database does not currently have the decision-matrix tables described in the architecture direction. Current proposal approval is reviewer approval, not a decision-matrix vote. The V3 architecture document is direction for future schema and policy; it is not proof that those tables are deployed.

3. How the initial database was built

Most of the initial canonical database came from retained Forgejo-era workspace files, not from another live Forgejo database being queried on every answer:

Forgejo-era file workspace
-> retained inventory/import runs
-> kb_stage staged rows
-> canonical UUID mappings
-> public claims, sources, evidence, and edges
-> current PostgreSQL database

The database retains enough mappings to account for most initial rows:

  • 1,807 of 1,837 claims map to retained staged imports;
  • all 4,145 canonical sources have canonical mappings;
  • 4,254 of 4,670 evidence links are directly reproduced by retained joins;
  • 4,878 of 4,916 edge rows are accounted for by staged relations.

The remaining historical rows need explicit genesis records, source-synthesis rules, or later-change receipts. This is why snapshot recovery is complete while full source-derived recompilation remains partial.

4. How Leo reads knowledge now

For database-dependent questions, the runtime compiles fresh facts from Postgres before the model answers. The validator then checks the draft against the same receipt. Unsupported state claims fail closed instead of being accepted because they sound plausible.

This now covers:

  • current counts and schema boundaries;
  • relevant claims, sources, evidence, and graph edges;
  • proposal status and applied_at;
  • named and versioned proposal packets;
  • reviewer approval versus canonical apply;
  • source-intake limits;
  • runtime memory versus canonical provenance;
  • handler execution versus Telegram-visible delivery.

5. How Leo changes knowledge now

The bounded source path is:

private source artifact
-> duplicate retrieval against canonical knowledge
-> up to three atomic candidates
-> exact source-fragment binding
-> provenance and uncertainty manifest
-> deterministic strict proposal payload
-> pending_review
-> human review
-> guarded apply
-> exact canonical row readback
-> private replay receipt

Leo can propose and prepare. It does not self-approve or silently self-apply. The separate kb_review and kb_apply capabilities enforce the review/apply boundary at the database level.

After a guarded apply, the operator now writes a private mode-0600 receipt that binds:

  • the immutable reviewed payload;
  • reviewer and applier metadata;
  • the exact executed apply-SQL hash;
  • exact canonical rows, including generated IDs and timestamps;
  • per-row, per-table, payload, and replay-material hashes.

If the file write is lost after commit, --receipt-only can recover the current row-level receipt without reapplying, provided all payload-controlled fields still match. A recovered engine hash is labeled as reconstructed, not falsely presented as original execution proof.

Work Completed

Expected behavior became testable

The vague report that “Leo is broken” was converted into separate proof tiers:

  • database truth;
  • hidden handler execution;
  • Telegram-visible delivery;
  • same-session memory;
  • proposal preparation and staging;
  • reviewer approval;
  • canonical apply;
  • restart persistence;
  • GCP parity.

This prevents a unit test, a handler response, a Telegram message, and a database mutation from being treated as the same result.

Broad reasoning was repaired

An early blind run exposed schema invention, incorrect edge semantics, memory used as provenance, overly long answers, and participant-name leakage. Those failures became runtime contracts and regression tests.

The current live VPS no-post GatewayRunner suite passes 15/15. It covers broad m3taversal-style questions about database composition, source intake, identity, restart truth, evidence linkage, memory, runtime versus canonical state, shared claims versus positions, forecast resolution, supersession, and uncertainty.

The V3 document became a real lifecycle canary

The V3 identity architecture document exposed and led to fixes for:

  • non-byte-exact model quotes;
  • line IDs that could point to blank lines;
  • exact source wording being confused with atomic canonical claim text;
  • named/versioned proposal questions routing to the wrong state readback;
  • a valid database-grounded answer being rejected by the response validator.

The corrected proposal is 10bc0719-1c2b-5f42-a41e-86e6478692cb, status pending_review, with database applied_at: NULL. Its candidates are not canonical and no production apply ran.

Restart survival was proven

After the managed restart, the exact V3 question still returned current DB counts and the correct proposal boundary. The gateway remains active with PID 366950, NRestarts=0, and start time 2026-07-14 00:25:49 UTC.

The restart proof was captured at runtime merge aa1246c603020bef07a575e4aebf4f2d2ed5db0c. Later operator-script and documentation deliveries through c7d91b679bd69ee2d1545ee7e169f7c2585d71c2 did not restart the gateway.

Exact database recovery was proven

The newest post-V3 dump contains 39 tables and 52,167 rows with 29 proposals. It restored into a fresh network-isolated PostgreSQL container with:

  • zero schema, constraint, role, row, hash, or performance mismatch;
  • 2.71 seconds for pg_restore;
  • 10.56 seconds total verification time;
  • no persistent storage;
  • no production DB contact;
  • complete container cleanup.

This is a real on-demand recovery path today.

Future apply provenance stopped getting worse

PRs #136 and #137 added, merged, deployed, and live-proved private replay receipts. The deployed command ran as the real teleo service user against an existing strict applied proposal in read-only recovery mode. It recovered one exact edge, wrote a teleo:teleo mode-600 receipt, changed no DB counts, left the gateway unchanged, and removed the private receipt afterward.

What Is Achieved

Outcome Evidence tier Status
Broad VPS database reasoning live VPS no-post handler, 15/15 Achieved
Current claim/source/evidence/proposal readback live VPS DB-grounded handler Achieved
Same-session memory without false DB provenance live VPS handler Achieved
Exact m3taversal naming live VPS handler and regression tests Achieved
Restart persistence managed restart plus post-restart V3 readback Achieved
One-document source preparation live VPS private source path Achieved
Grounded proposal staging live VPS pending_review row Achieved
Exact snapshot rebuild isolated local Postgres, 52,167 rows Achieved
New strict-apply row receipt deployed VPS read-only recovery Achieved
Current disposable GCP copy and ID-free replay private Cloud SQL and GCP compute, 52,167/52,167 rows Achieved and cleaned up
Genesis plus strict insert-only replay isolated local Postgres through merged PR #146 Achieved for supported strict contracts

What Is Not Achieved

Missing outcome Why it is still open
Current broad Telegram-visible proof Telegram Web in the connected Google Chrome is not authenticated; handler proof is not visible-chat proof
V3 production canonical apply proposal remains pending_review; no review or apply was authorized
Arbitrary automatic document/post/tweet intake the current source command handles one bounded text-like artifact; general fetch/extract/stage automation is not complete
Persistent GCP parity and promotion the disposable 52,167-row proof passed, but persistent teleo_canonical remains the older staging copy and is not cut over
Full source-to-blank-DB recompilation historical provenance gaps and unsupported mutating contracts remain beyond the merged insert-only genesis-plus-ledger slice
General DB-to-identity rendering no complete scheduled DB-to-SOUL.md renderer is proven
Deployed decision matrix matrix tables and voting flow are absent from the current live schema

How This Moves The VPS Forward

  • Leo relies less on repeatedly trained answer style and more on current DB readback.
  • Broad questions work without the user supplying claim IDs.
  • Proposed, approved, and applied knowledge are visibly separated.
  • Source preparation has exact provenance instead of model-only summaries.
  • Restart behavior and database-backed answers are repeatable.
  • Future approved changes can carry row-level replay material instead of becoming another historical mystery.

How This Moves GCP Forward

  • The current post-V3 snapshot has been restored to a disposable private Cloud SQL clone with exact parity, bounded no-send replay, and cleanup.
  • Persistent teleo_canonical remains the older staging copy; promotion, continuous replication, Telegram delivery, and production cutover are separate decisions and proof rows.
  • Open PR #148 proposes least-privilege runtime access. Its branch and proposed live end state remain candidate evidence until merge and receipt verification.

How This Moves On-Demand Rebuild Forward

There are now two distinct capabilities:

  1. Exact recovery, working now: restore the verified snapshot and compare schema, roles, hashes, rows, and performance.
  2. Semantic recompilation, still partial: rebuild from immutable sources, deterministic extraction, reviewed payloads, and accepted row receipts.

The practical architecture is:

verified post-V3 snapshot = genesis epoch
+ every later accepted strict payload
+ every later row-level replay receipt
-> deterministic replay
-> canonical manifest comparison
-> restart
-> broad answer benchmark

The new receipt path closes the future side of this model. Merged PR #146 now replays supported strict insert-only receipts on a clean genesis clone and proves exact post-apply parity. The next reconstruction work is to retain complete deltas for mutating contracts, handle or reject legacy freeform applies, and repair historical import gaps without allowing new gaps to accumulate.

Delivery Chain

The latest coherent delivery sequence is merged:

PR Outcome
#127 VPS document preparation and review staging
#128 byte-exact source references
#129 nonempty dense source-fragment IDs
#130 atomic claims separated from exact quotes
#131 retained V3 lifecycle proof
#132 named/versioned proposal routing
#133 response-validator compatibility
#134 three-day VPS/GCP/rebuild handoff
#135 current post-V3 snapshot and exact rebuild proof
#136 private replay receipts for guarded applies
#137 deployed receipt runtime proof
#144 GCP database-first restore, fail-closed helper, and live restart proof
#146 unified turn manifest and deterministic genesis-plus-ledger replay
#147 unseen-reasoning verifier hardened for valid reasoning variance

PR #148 is open candidate work and is intentionally excluded from the merged delivery chain above.

Immediate Next Outcomes

  1. Authenticate Telegram Web in the connected Google Chrome and run visible, out-of-sample m3taversal-style questions.
  2. Review and merge or reject PR #148, then verify the least-privilege runtime receipt before considering any persistent GCP promotion.
  3. Extend genesis-plus-receipt reconstruction beyond supported insert-only contracts while retaining exact before/after deltas.
  4. Run one explicitly reviewed source packet through guarded apply in a disposable clone before seeking any production apply authorization.
  5. Reconstruct the remaining historical source, evidence, and edge provenance.

Evidence

Final Claim Ceiling

The last three days produced a stable, testable, database-grounded Leo on the VPS at the handler tier, a real source-to-review path, exact snapshot recovery, and durable replay evidence for future strict applies. Subsequent work added disposable current GCP parity plus no-send replay and an exact insert-only genesis-plus-ledger slice. The system still lacks current Telegram-visible broad proof, persistent GCP parity/promotion, a V3 production apply, general automatic source ingestion, and full historical source-derived database recompilation.