Merge pull request #138 from living-ip/codex/leo-architecture-outcomes-summary-20260714

Document Leo architecture, work, and outcomes
2026-07-14 03:56:20 +02:00 · 2026-07-14 03:56:20 +02:00 · dcb3a83f00
commit dcb3a83f00
parent c7d91b679b af13c0b76a
2 changed files with 376 additions and 3 deletions
--- a/docs/reports/leo-working-state-20260709/current-truth-index.md
+++ b/docs/reports/leo-working-state-20260709/current-truth-index.md
@ -8,10 +8,13 @@ The current canonical handoff is
 `working-leo-three-day-outcome-and-forward-path-20260714.md`. This section
 supersedes the July 13 verdict below.

+For the architecture and outcome-level summary, read
+`leo-architecture-work-and-outcomes-20260714.md` first.
+
 - VPS gateway behavior was restart-proven at merge
-  `aa1246c603020bef07a575e4aebf4f2d2ed5db0c`; the deploy-infra workspace is now
-  at operator-script merge `7634fc9208f1b43a0037d6c2e246112e18c841fd`.
-  The script-only delivery did not restart the gateway: PID `366950`,
+  `aa1246c603020bef07a575e4aebf4f2d2ed5db0c`. Later operator-script and proof
+  deliveries through `c7d91b679bd69ee2d1545ee7e169f7c2585d71c2` did not
+  restart the gateway: PID `366950`,
  `NRestarts=0`, active/running, start `2026-07-14 00:25:49 UTC`.
 - Live counts are claims `1837`, sources `4145`, edges `4916`, evidence `4670`,
  proposals `29`; proposal states are applied `2`, approved `3`, pending review
@ -37,6 +40,7 @@ supersedes the July 13 verdict below.

 Newest artifacts:

+- `leo-architecture-work-and-outcomes-20260714.md`
 - `working-leo-three-day-outcome-and-forward-path-20260714.md`
 - `telegram-handler-m3taversal-oos-suite-score-current.md`
 - `telegram-handler-v3-proposal-readback-post-restart-current.json`
--- a/docs/reports/leo-working-state-20260709/leo-architecture-work-and-outcomes-20260714.md
+++ b/docs/reports/leo-working-state-20260709/leo-architecture-work-and-outcomes-20260714.md
@ -0,0 +1,369 @@
+# Leo Architecture, Work, And Outcomes
+
+Current as of `2026-07-14 01:40 UTC`.
+
+This is the executive summary of what Leo is, how its knowledge system works,
+what changed during the July 10-14 delivery wave, what is genuinely working,
+and what remains incomplete on the VPS, GCP, and database-rebuild paths.
+
+## Plain-Language Verdict
+
+**Leo on the VPS is materially working at the live database-grounded handler
+tier.** It can find current knowledge without being handed row IDs, distinguish
+proposals from live knowledge, remember the current conversation, prepare a
+document into grounded candidates, stage a review proposal, explain uncertainty,
+use only the visible handle `m3taversal`, and preserve this behavior across a
+managed gateway restart. The current broad no-post suite passes `15/15`.
+
+**The whole system is not complete.** The current broad suite has not yet been
+repeated visibly in the real Telegram group through authenticated Google Chrome.
+The newest post-V3 database copy has not yet been restored and replayed on GCP.
+No V3 production proposal has been canonically applied. A snapshot can rebuild
+the current database exactly, but the entire historical database cannot yet be
+recompiled from raw documents and review history alone.
+
+## The Architecture
+
+```mermaid
+flowchart TD
+    U["Telegram user"] --> T["Telegram gateway"]
+    T --> H["Hermes / Leo runtime"]
+    H --> C["Fresh DB context compiler"]
+    C --> P[("Canonical Postgres in VPS Docker")]
+    H --> V["Response validator"]
+    P --> V
+    V --> T
+
+    D["Document, post, transcript, or extraction"] --> S["Private source preparation"]
+    S --> X["Exact source fragments and candidate claims"]
+    X --> Q["kb_stage proposal"]
+    Q --> R["Human review"]
+    R --> A["Guarded kb_apply operator"]
+    A --> P
+    A --> L["Private row-level replay receipt"]
+
+    P --> B["Verified snapshot and manifest"]
+    B --> DR["Exact restore on demand"]
+    B --> G["GCP staging copy"]
+    L --> SR["Genesis plus accepted-ledger replayer"]
+```
+
+### 1. User and runtime layer
+
+- Telegram is the user-facing surface.
+- The gateway sends the message to the Hermes-based Leo runtime.
+- Same-session conversation memory belongs to the runtime, not to canonical
+  knowledge.
+- Runtime files such as profile state or `SOUL.md` can shape behavior, but they
+  are not the authoritative claim and evidence database.
+
+### 2. Database truth layer
+
+The live canonical database is PostgreSQL in the `teleo-pg` Docker container on
+the VPS. Its current key counts are:
+
+| Data | Rows |
+| --- | ---: |
+| Claims | `1,837` |
+| Sources | `4,145` |
+| Claim-evidence links | `4,670` |
+| Claim edges | `4,916` |
+| Reasoning tools | `17` |
+| Proposals | `29` |
+
+The important ownership boundary is:
+
+- `public.*` is canonical, live knowledge;
+- `kb_stage.*` is imports, mappings, proposals, review state, and apply state;
+- runtime memory and workspace files are supporting state, not canonical DB
+  proof;
+- a proposal is not live knowledge merely because it exists or is approved;
+- canonical apply requires committed `public.*` rows, proposal status `applied`,
+  non-null `applied_at`, and row-level postflight readback.
+
+The live database does not currently have the decision-matrix tables described
+in the architecture direction. Current proposal approval is reviewer approval,
+not a decision-matrix vote. The V3 architecture document is direction for future
+schema and policy; it is not proof that those tables are deployed.
+
+### 3. How the initial database was built
+
+Most of the initial canonical database came from retained Forgejo-era workspace
+files, not from another live Forgejo database being queried on every answer:
+
+```text
+Forgejo-era file workspace
+-> retained inventory/import runs
+-> kb_stage staged rows
+-> canonical UUID mappings
+-> public claims, sources, evidence, and edges
+-> current PostgreSQL database
+```
+
+The database retains enough mappings to account for most initial rows:
+
+- `1,807` of `1,837` claims map to retained staged imports;
+- all `4,145` canonical sources have canonical mappings;
+- `4,254` of `4,670` evidence links are directly reproduced by retained joins;
+- `4,878` of `4,916` edge rows are accounted for by staged relations.
+
+The remaining historical rows need explicit genesis records, source-synthesis
+rules, or later-change receipts. This is why snapshot recovery is complete while
+full source-derived recompilation remains partial.
+
+### 4. How Leo reads knowledge now
+
+For database-dependent questions, the runtime compiles fresh facts from
+Postgres before the model answers. The validator then checks the draft against
+the same receipt. Unsupported state claims fail closed instead of being accepted
+because they sound plausible.
+
+This now covers:
+
+- current counts and schema boundaries;
+- relevant claims, sources, evidence, and graph edges;
+- proposal status and `applied_at`;
+- named and versioned proposal packets;
+- reviewer approval versus canonical apply;
+- source-intake limits;
+- runtime memory versus canonical provenance;
+- handler execution versus Telegram-visible delivery.
+
+### 5. How Leo changes knowledge now
+
+The bounded source path is:
+
+```text
+private source artifact
+-> duplicate retrieval against canonical knowledge
+-> up to three atomic candidates
+-> exact source-fragment binding
+-> provenance and uncertainty manifest
+-> deterministic strict proposal payload
+-> pending_review
+-> human review
+-> guarded apply
+-> exact canonical row readback
+-> private replay receipt
+```
+
+Leo can propose and prepare. It does not self-approve or silently self-apply.
+The separate `kb_review` and `kb_apply` capabilities enforce the review/apply
+boundary at the database level.
+
+After a guarded apply, the operator now writes a private mode-`0600` receipt
+that binds:
+
+- the immutable reviewed payload;
+- reviewer and applier metadata;
+- the exact executed apply-SQL hash;
+- exact canonical rows, including generated IDs and timestamps;
+- per-row, per-table, payload, and replay-material hashes.
+
+If the file write is lost after commit, `--receipt-only` can recover the current
+row-level receipt without reapplying, provided all payload-controlled fields
+still match. A recovered engine hash is labeled as reconstructed, not falsely
+presented as original execution proof.
+
+## Work Completed
+
+### Expected behavior became testable
+
+The vague report that “Leo is broken” was converted into separate proof tiers:
+
+- database truth;
+- hidden handler execution;
+- Telegram-visible delivery;
+- same-session memory;
+- proposal preparation and staging;
+- reviewer approval;
+- canonical apply;
+- restart persistence;
+- GCP parity.
+
+This prevents a unit test, a handler response, a Telegram message, and a
+database mutation from being treated as the same result.
+
+### Broad reasoning was repaired
+
+An early blind run exposed schema invention, incorrect edge semantics, memory
+used as provenance, overly long answers, and participant-name leakage. Those
+failures became runtime contracts and regression tests.
+
+The current live VPS no-post GatewayRunner suite passes `15/15`. It covers broad
+m3taversal-style questions about database composition, source intake, identity,
+restart truth, evidence linkage, memory, runtime versus canonical state, shared
+claims versus positions, forecast resolution, supersession, and uncertainty.
+
+### The V3 document became a real lifecycle canary
+
+The V3 identity architecture document exposed and led to fixes for:
+
+- non-byte-exact model quotes;
+- line IDs that could point to blank lines;
+- exact source wording being confused with atomic canonical claim text;
+- named/versioned proposal questions routing to the wrong state readback;
+- a valid database-grounded answer being rejected by the response validator.
+
+The corrected proposal is
+`10bc0719-1c2b-5f42-a41e-86e6478692cb`, status `pending_review`, with database
+`applied_at: NULL`. Its candidates are not canonical and no production apply ran.
+
+### Restart survival was proven
+
+After the managed restart, the exact V3 question still returned current DB
+counts and the correct proposal boundary. The gateway remains active with PID
+`366950`, `NRestarts=0`, and start time `2026-07-14 00:25:49 UTC`.
+
+The restart proof was captured at runtime merge
+`aa1246c603020bef07a575e4aebf4f2d2ed5db0c`. Later operator-script and
+documentation deliveries through
+`c7d91b679bd69ee2d1545ee7e169f7c2585d71c2` did not restart the gateway.
+
+### Exact database recovery was proven
+
+The newest post-V3 dump contains `39` tables and `52,167` rows with `29`
+proposals. It restored into a fresh network-isolated PostgreSQL container with:
+
+- zero schema, constraint, role, row, hash, or performance mismatch;
+- `2.71` seconds for `pg_restore`;
+- `10.56` seconds total verification time;
+- no persistent storage;
+- no production DB contact;
+- complete container cleanup.
+
+This is a real on-demand recovery path today.
+
+### Future apply provenance stopped getting worse
+
+PRs `#136` and `#137` added, merged, deployed, and live-proved private replay
+receipts. The deployed command ran as the real `teleo` service user against an
+existing strict applied proposal in read-only recovery mode. It recovered one
+exact edge, wrote a `teleo:teleo` mode-`600` receipt, changed no DB counts,
+left the gateway unchanged, and removed the private receipt afterward.
+
+## What Is Achieved
+
+| Outcome | Evidence tier | Status |
+| --- | --- | --- |
+| Broad VPS database reasoning | live VPS no-post handler, `15/15` | Achieved |
+| Current claim/source/evidence/proposal readback | live VPS DB-grounded handler | Achieved |
+| Same-session memory without false DB provenance | live VPS handler | Achieved |
+| Exact `m3taversal` naming | live VPS handler and regression tests | Achieved |
+| Restart persistence | managed restart plus post-restart V3 readback | Achieved |
+| One-document source preparation | live VPS private source path | Achieved |
+| Grounded proposal staging | live VPS `pending_review` row | Achieved |
+| Exact snapshot rebuild | isolated local Postgres, `52,167` rows | Achieved |
+| New strict-apply row receipt | deployed VPS read-only recovery | Achieved |
+| Prior GCP database copy and six-question replay | private Cloud SQL and GCP compute | Achieved at the earlier `52,164`-row proof point |
+
+## What Is Not Achieved
+
+| Missing outcome | Why it is still open |
+| --- | --- |
+| Current broad Telegram-visible proof | Telegram Web in the connected Google Chrome is not authenticated; handler proof is not visible-chat proof |
+| V3 production canonical apply | proposal remains `pending_review`; no review or apply was authorized |
+| Current GCP parity | the `52,167`-row post-V3 snapshot has not yet been restored and replayed on GCP |
+| Arbitrary automatic document/post/tweet intake | the current source command handles one bounded text-like artifact; general fetch/extract/stage automation is not complete |
+| Full source-to-blank-DB recompilation | historical provenance gaps remain and the genesis-plus-ledger replay runner is not finished |
+| General DB-to-identity rendering | no complete scheduled DB-to-`SOUL.md` renderer is proven |
+| Deployed decision matrix | matrix tables and voting flow are absent from the current live schema |
+
+## How This Moves The VPS Forward
+
+- Leo relies less on repeatedly trained answer style and more on current DB
+  readback.
+- Broad questions work without the user supplying claim IDs.
+- Proposed, approved, and applied knowledge are visibly separated.
+- Source preparation has exact provenance instead of model-only summaries.
+- Restart behavior and database-backed answers are repeatable.
+- Future approved changes can carry row-level replay material instead of becoming
+  another historical mystery.
+
+## How This Moves GCP Forward
+
+- The prior GCP vertical slice proved the architecture can run against private
+  Cloud SQL and real GCP compute.
+- The current post-V3 dump and manifest are already captured and locally
+  verified, so GCP work starts from a known input rather than rebuilding one.
+- The same `15/15` suite and exact V3 question are ready for replay.
+- The remaining GCP work is operational: authenticate, restore the current copy,
+  verify parity/private connectivity/performance, replay, and clean up.
+
+## How This Moves On-Demand Rebuild Forward
+
+There are now two distinct capabilities:
+
+1. **Exact recovery, working now:** restore the verified snapshot and compare
+   schema, roles, hashes, rows, and performance.
+2. **Semantic recompilation, still partial:** rebuild from immutable sources,
+   deterministic extraction, reviewed payloads, and accepted row receipts.
+
+The practical architecture is:
+
+```text
+verified post-V3 snapshot = genesis epoch
+ every later accepted strict payload
+ every later row-level replay receipt
+-> deterministic replay
+-> canonical manifest comparison
+-> restart
+-> broad answer benchmark
+```
+
+The new receipt path closes the future side of this model. The next repo-owned
+step is to replay one reviewed `approve_claim` receipt onto a clean genesis clone
+and prove exact post-apply parity. Historical import gaps can then be repaired
+without allowing new gaps to accumulate.
+
+## Delivery Chain
+
+The latest coherent delivery sequence is merged:
+
+| PR | Outcome |
+| --- | --- |
+| `#127` | VPS document preparation and review staging |
+| `#128` | byte-exact source references |
+| `#129` | nonempty dense source-fragment IDs |
+| `#130` | atomic claims separated from exact quotes |
+| `#131` | retained V3 lifecycle proof |
+| `#132` | named/versioned proposal routing |
+| `#133` | response-validator compatibility |
+| `#134` | three-day VPS/GCP/rebuild handoff |
+| `#135` | current post-V3 snapshot and exact rebuild proof |
+| `#136` | private replay receipts for guarded applies |
+| `#137` | deployed receipt runtime proof |
+
+## Immediate Next Outcomes
+
+1. Authenticate Telegram Web in the connected Google Chrome and run visible,
+   out-of-sample m3taversal-style questions.
+2. Authenticate Google Cloud CLI through the connected Chrome flow, restore the
+   current post-V3 snapshot to disposable GCP staging, verify parity, replay the
+   broad suite, and remove temporary resources.
+3. Complete the genesis-plus-receipt replayer for `approve_claim` bundles and
+   prove exact clone-to-replay parity.
+4. Run one explicitly reviewed source packet through guarded apply in a
+   disposable clone before seeking any production apply authorization.
+5. Reconstruct the remaining historical source, evidence, and edge provenance.
+
+## Evidence
+
+- [Current truth index](current-truth-index.md)
+- [Three-day outcome and forward path](working-leo-three-day-outcome-and-forward-path-20260714.md)
+- [Database rebuild and recompilation](../../kb-rebuild-and-recompile.md)
+- [Broad VPS suite score](telegram-handler-m3taversal-oos-suite-score-current.md)
+- [Post-restart V3 readback](telegram-handler-v3-proposal-readback-post-restart-current.json)
+- [V3 document lifecycle](leo-v3-document-source-lifecycle-current.md)
+- [Post-V3 snapshot](gcp-refresh-post-v3-snapshot-current.json)
+- [Post-V3 exact local rebuild](gcp-refresh-post-v3-local-rebuild-current.json)
+- [Deployed replay-receipt proof](kb-apply-replay-receipt-current.json)
+
+## Final Claim Ceiling
+
+The last three days produced a stable, testable, database-grounded Leo on the
+VPS at the handler tier, a real source-to-review path, exact snapshot recovery,
+and durable replay evidence for future strict applies. They did not complete
+current Telegram-visible broad proof, current GCP parity, a V3 production apply,
+general automatic source ingestion, or full historical source-derived database
+recompilation.