210 lines
12 KiB
Markdown
210 lines
12 KiB
Markdown
---
|
|
name: working-leo-m3taversal-outcomes
|
|
description: "Use when defining, testing, or repairing Leo against m3taversal's expected outcomes: Telegram memory, critical reasoning, canonical KB truth, proposed-vs-approved-vs-applied state, and guarded DB manipulation."
|
|
---
|
|
|
|
# Working Leo / m3taversal Outcomes
|
|
|
|
## Job
|
|
|
|
Keep Leo work anchored to what m3taversal appears to mean by "working": not just answers, but remembered context, truthful KB state, and approved concrete changes becoming canonical rows through a guarded proof path.
|
|
|
|
## Participant Identity Rule
|
|
|
|
- Address `@m3taversal` only as `m3taversal`, exactly.
|
|
- Never infer, invent, shorten, or substitute a personal name from a session
|
|
header, memory, identity document, another chat, or another participant.
|
|
- Resolve the speaker from the current Telegram update. Do not transfer identity
|
|
across users when someone replies, tags an account, or joins the thread.
|
|
- The legacy database value `m3ta` may be reported only as a quoted stored
|
|
reviewer handle. It is never a form of address.
|
|
- Standard response labels are neutral. Use `Next proof-changing follow-up:`.
|
|
|
|
## Trigger Phrases
|
|
|
|
- "working Leo"
|
|
- "m3taversal expected outcomes"
|
|
- "m3taversal says Leo is broken"
|
|
- "able to manipulate the knowledge base"
|
|
- "same state as last night"
|
|
- "critical reasoning behavior"
|
|
|
|
## Definition Of Working
|
|
|
|
A working Leo is a Telegram-facing agent that:
|
|
|
|
1. Remembers the current operator conversation.
|
|
2. Grounds KB answers in canonical Teleo Postgres rows when claiming KB truth.
|
|
3. Distinguishes `proposed`, `pending_review`, `approved`, `applied`, and `not applied`.
|
|
4. Does not say an approval changed canonical DB state when only `kb_stage` changed.
|
|
5. Can answer vague, high-level m3taversal-style incident prompts without being spoon-fed exact proposal IDs.
|
|
6. Can compose the KB from a previously unindexed document, URL, or tweet-like
|
|
source: retain a byte/hash-bound source locator, extract atomic claims and
|
|
exact evidence excerpts, preserve useful metadata, and link every claim to
|
|
its evidence and source.
|
|
7. Detects duplicates, conflicts, updates, and insufficient evidence before
|
|
staging; uncertainty must remain visible instead of being normalized into a
|
|
stronger claim.
|
|
8. Can stage those concrete KB changes from Telegram with enough structure for
|
|
human review, without making staged content canonical.
|
|
9. Can move approved concrete changes through a guarded apply path when authorized.
|
|
10. Retains m3taversal's caveats and review notes in source/evidence/proposal rows.
|
|
11. Reasons over claims, evidence, sources, edges, and open conflicts as a graph,
|
|
and can explain which rows support or weaken an answer without being given IDs.
|
|
12. Rebuilds any compiled identity/workspace artifact deterministically from
|
|
canonical DB rows, reports the source rows and freshness boundary, and does
|
|
not treat edits to a generated artifact as canonical knowledge.
|
|
13. Produces before/after table-level proof and service stability readback.
|
|
14. Survives an intentional `leoclean-gateway.service` restart: active before, active after, canonical KB counts unchanged, and a no-post handler smoke still answers.
|
|
15. For a reviewed graph bundle, produces exact payload-controlled row
|
|
projections, exact table deltas, source-byte binding, and cleanup proof in a
|
|
disposable runtime before any production permission or apply window.
|
|
|
|
## Current Benchmark Cases
|
|
|
|
- Marker memory: `WL-LIVE-TG-CORY-20260709`.
|
|
- Truth correction: `approved != applied`.
|
|
- Applied canary: proposal `00957f6c-9883-4015-95a4-6b09367efb0e` and edge `c167933e-d513-4f43-9335-d5d8aeb259f2`.
|
|
- Staged write canary: proposal `8dfedb3f-3aa4-4200-970f-4c0016f6869f`, status `pending_review`, with no new public canonical rows after the test timestamp.
|
|
- Open-ended triage canary: `WL-LIVE-TG-CORY-OPEN-20260709`, where Leo inferred the likely failure mode from "agents not working / same state as last night" and explained the proposed, approved, and applied split without exact IDs.
|
|
- Old rich proposal packets: `14fa5ecc...`, `ac036c9d...`, and `a64df080...` are not to be silently production-applied.
|
|
- Guarded apply proof: both generic and real Helmer v3 receipts pass `37/37` in
|
|
disposable PostgreSQL; production Helmer remains unapplied.
|
|
- Clone-bound handler proof: the real VPS `GatewayRunner` can inspect the full
|
|
current-data disposable clone, discover the Helmer proposal, distinguish
|
|
`approved` from `applied`, and produce bound bridge-call evidence without a
|
|
Telegram post or production change. The proof surface must expose no delivery
|
|
adapters or send tool, restrict terminal execution to clone-bound read-only KB
|
|
verbs, kill the handler process group on timeout, and compare production
|
|
row-content fingerprints before and after. Treat open-ended latency and
|
|
retries as a separate reliability dimension; a correct answer after a failed
|
|
tool call is recovered behavior, not a clean pass, and one correct answer does
|
|
not establish a stable pass rate.
|
|
- Source-composition proof: the real VPS `GatewayRunner` passed `34/34` in a
|
|
fresh no-send full-data clone. It searched existing knowledge, extracted two
|
|
hash-bound conflicting claims from a new document and post, staged a strict
|
|
proposal, preserved an immutable separated approval, applied exact canonical
|
|
rows, reopened in a new child process, recalled the prior marker, discovered
|
|
the new proposal/claims without supplied IDs, read exact evidence/source UUIDs
|
|
and edges, and explained `approved != applied`. Production DB fingerprints,
|
|
service PID/restarts, and live bridge hashes stayed unchanged; all disposable
|
|
containers, volumes, profiles, credentials, and run directories were removed.
|
|
This is clone proof, not Telegram delivery or production apply proof.
|
|
|
|
## Direct Questions
|
|
|
|
For vague or no-context questions, answer directly and then name the one action
|
|
that would change the proof:
|
|
|
|
- "Did the DB change?": split applied rows, approved-but-unapplied proposals,
|
|
pending rows, and canceled rows. Say `Approved is not the same as applied`.
|
|
- "Is Helmer in Leo?": say `no, not canonical` until the exact claims, sources,
|
|
evidence, edges, reasoning tool, and applied ledger read back from production.
|
|
- "Did the decision matrix approve it?": verify matrix tables first; reviewer
|
|
approval is not a matrix vote.
|
|
- "Are document pointers the blocker?": answer `not just pointer mismatch` and
|
|
separate files, source refs, canonical source rows, review, and apply.
|
|
- "Can I demo KB mutation?": split staging-demo truth from canonical-apply truth.
|
|
- "Did editing SOUL.md change identity?": no canonical identity change without
|
|
row IDs plus render/sync postflight.
|
|
|
|
End no-context answers with exactly one `Next proof-changing follow-up:` line.
|
|
|
|
## Current Verdict - 2026-07-13
|
|
|
|
Use `not fully yet` for the whole m3taversal-standard question until every row below
|
|
is green at its required tier:
|
|
|
|
- VPS runtime and restart survival: proven.
|
|
- VPS no-send `DC-01` through `DC-06`: three post-deploy clean-session trials
|
|
strict-score `6/6` (`18/18` replies) with unchanged DB/service state and
|
|
complete temporary-profile cleanup.
|
|
- Telegram-visible open-ended read-only behavior and conversation memory:
|
|
proven for the earlier `OE-01` through `OE-05` suite, but that narrower suite
|
|
does not establish current broad reliability.
|
|
- Telegram-visible no-context direct-claim suite: all six prompts/replies are
|
|
captured with unchanged DB/service state, but the hardened score is `5/6`.
|
|
`DC-05` proposes an incompatible legacy `add_edge` apply target.
|
|
- Canonical apply primitive and approved bundle lifecycle: proven in isolation;
|
|
broad production packet application is not authorized or executed.
|
|
- Source composition: deterministic fixture proof plus a full-data, no-send,
|
|
current-VPS clone checkpoint are proven. Arbitrary production document/tweet
|
|
ingestion is not proven.
|
|
- Broad blind VPS behavior: operationally clean but semantically failed.
|
|
Twelve of twelve prompts returned, DB/service state stayed unchanged, and the
|
|
temporary profile was removed; independent strict judges accepted only
|
|
`1/12` and `2/12` outright. Failures include invented current schema fields
|
|
and edge types, handler proof described as Telegram-live, incorrect runtime
|
|
memory boundaries, and temporary memory treated as source provenance.
|
|
- GCP canonical DB parity and six DB-read routes: proven. A later adapter-free
|
|
model replay passed `6/6` with exact count consistency and unchanged state.
|
|
Durable on-demand operator access and cleanup of the retained replay clone
|
|
remain open because the expected operator service accounts are absent and the
|
|
privileged Google account requires human password/MFA reauthentication.
|
|
|
|
Do not answer `yes` merely because all repo tests pass. The final user-facing
|
|
proof is a visible Telegram conversation plus truthful canonical row readback.
|
|
|
|
## Hard m3taversal Question Bank
|
|
|
|
Use these without IDs, schema hints, or guardrail-heavy setup. The answer must
|
|
infer the relevant rows and end with one proof-changing follow-up:
|
|
|
|
1. `I approved Helmer weeks ago. Why isn't it real yet?`
|
|
2. `What can I show a partner in five minutes?`
|
|
3. `This PDF should update Leo. What happens now?`
|
|
4. `What changed in your identity since last week?`
|
|
5. `Make this claim canonical: <claim text>.`
|
|
6. `Are pending proposals stuck because sources are wrong?`
|
|
7. `What does our strategy depend on, and which evidence could overturn it?`
|
|
8. `What's the next KB change I should approve?`
|
|
|
|
Score the answer on directness, fresh canonical lookup, claim/evidence/source
|
|
reasoning, state semantics, uncertainty, row-level proof, and the usefulness of
|
|
the next action. Penalize asking m3taversal for IDs that Leo can discover itself.
|
|
|
|
Also fail the answer when it presents proposed architecture as current v1. The
|
|
current `public.claims` table has `text` and `superseded_by`, not `body`, generic
|
|
metadata, or forecast-resolution fields. Current `public.sources` has no
|
|
author/channel/date columns. Current accepted edge types are `supports`,
|
|
`challenges`, `requires`, `relates`, `contradicts`, `supersedes`,
|
|
`derives_from`, `cites`, `causes`, `constrains`, and `accelerates`.
|
|
|
|
## Required Answer Discipline
|
|
|
|
When Leo or a worker answers about KB state, it must say:
|
|
|
|
- What source was used: memory, Telegram history, `kb_stage`, canonical `public.*`, or filesystem/runtime.
|
|
- Which rows/tables changed.
|
|
- Which proposal IDs are only staged/approved.
|
|
- Which canonical rows exist.
|
|
- What remains pending or deferred.
|
|
|
|
## Not Done
|
|
|
|
Leo is not working just because:
|
|
|
|
- the systemd service is active,
|
|
- Bot API calls work,
|
|
- a local handler canary passed,
|
|
- a proposal was approved,
|
|
- a packet was generated,
|
|
- a clone rehearsal passed.
|
|
- `NRestarts=0` was observed without an intentional restart-cycle proof.
|
|
|
|
Those are evidence. The target is Telegram-visible behavior plus DB-state truth plus a guarded path to canonical rows.
|
|
|
|
A synthetic one-claim proposal proves only the guarded mutation primitive. It
|
|
does not prove KB composition. Composition requires a source that was absent at
|
|
baseline, model-driven extraction into linked source/evidence/claim proposals,
|
|
dedupe/conflict readback, guarded canonical apply, and an open-ended answer
|
|
grounded in the new rows after an isolated handler restart.
|
|
|
|
## Proof Files
|
|
|
|
Start with `docs/reports/leo-working-state-20260709/working-leo-current-proof-20260712.md`
|
|
and its JSON companion for the current whole-system verdict and completion rule.
|
|
Use `docs/reports/leo-working-state-20260709/current-truth-index.md` for current artifact paths.
|
|
Use `docs/reports/leo-working-state-20260709/leo-source-composition-clone-checkpoint-current.md`
|
|
and its JSON companion for the current source-composition claim ceiling.
|
|
For restart survival, use `scripts/collect_leo_restart_survival_proof.py --execute-restart`; expected artifacts are `docs/reports/leo-working-state-20260709/leo-restart-survival-proof-current.json` and `.md`.
|