Some checks are pending
CI / lint-and-test (push) Waiting to run
* make Leo skill onboarding clean-context reproducible * harden Leo clean-context skill acceptance
220 lines
12 KiB
Markdown
220 lines
12 KiB
Markdown
---
|
|
name: working-leo-m3taversal-outcomes
|
|
description: "Use when defining, testing, or repairing Leo against m3taversal's expected outcomes: Telegram memory, critical reasoning, canonical KB truth, proposed-vs-approved-vs-applied state, and guarded DB manipulation."
|
|
---
|
|
|
|
# Working Leo / m3taversal Outcomes
|
|
|
|
## Job
|
|
|
|
Keep Leo work anchored to what m3taversal appears to mean by "working": not just answers, but remembered context, truthful KB state, and approved concrete changes becoming canonical rows through a guarded proof path.
|
|
|
|
## Participant Identity Rule
|
|
|
|
- Address `@m3taversal` only as `m3taversal`, exactly.
|
|
- Never infer, invent, shorten, or substitute a personal name from a session
|
|
header, memory, identity document, another chat, or another participant.
|
|
- Resolve the speaker from the current Telegram update. Do not transfer identity
|
|
across users when someone replies, tags an account, or joins the thread.
|
|
- The legacy database value `m3ta` may be reported only as a quoted stored
|
|
reviewer handle. It is never a form of address.
|
|
- Standard response labels are neutral. Use `Next proof-changing follow-up:`.
|
|
|
|
## Trigger Phrases
|
|
|
|
- "working Leo"
|
|
- "m3taversal expected outcomes"
|
|
- "m3taversal says Leo is broken"
|
|
- "able to manipulate the knowledge base"
|
|
- "same state as last night"
|
|
- "critical reasoning behavior"
|
|
|
|
## Definition Of Working
|
|
|
|
A working Leo is a Telegram-facing agent that:
|
|
|
|
1. Remembers the current operator conversation.
|
|
2. Grounds KB answers in canonical Teleo Postgres rows when claiming KB truth.
|
|
3. Distinguishes `proposed`, `pending_review`, `approved`, `applied`, and `not applied`.
|
|
4. Does not say an approval changed canonical DB state when only `kb_stage` changed.
|
|
5. Can answer vague, high-level m3taversal-style incident prompts without being spoon-fed exact proposal IDs.
|
|
6. Can compose the KB from a previously unindexed document, URL, or tweet-like
|
|
source: retain a byte/hash-bound source locator, extract atomic claims and
|
|
exact evidence excerpts, preserve useful metadata, and link every claim to
|
|
its evidence and source.
|
|
7. Detects duplicates, conflicts, updates, and insufficient evidence before
|
|
staging; uncertainty must remain visible instead of being normalized into a
|
|
stronger claim.
|
|
8. Can stage those concrete KB changes from Telegram with enough structure for
|
|
human review, without making staged content canonical.
|
|
9. Can move approved concrete changes through a guarded apply path when authorized.
|
|
10. Retains m3taversal's caveats and review notes in source/evidence/proposal rows.
|
|
11. Reasons over claims, evidence, sources, edges, and open conflicts as a graph,
|
|
and can explain which rows support or weaken an answer without being given IDs.
|
|
12. Rebuilds any compiled identity/workspace artifact deterministically from
|
|
canonical DB rows, reports the source rows and freshness boundary, and does
|
|
not treat edits to a generated artifact as canonical knowledge.
|
|
13. Produces before/after table-level proof and service stability readback.
|
|
14. Survives an intentional `leoclean-gateway.service` restart: active before, active after, canonical KB counts unchanged, and a no-post handler smoke still answers.
|
|
15. For a reviewed graph bundle, produces exact payload-controlled row
|
|
projections, exact table deltas, source-byte binding, and cleanup proof in a
|
|
disposable runtime before any production permission or apply window.
|
|
|
|
## Current Benchmark Cases
|
|
|
|
- Historical marker-memory receipt:
|
|
`docs/reports/leo-working-state-20260709/telegram-live-canary-current.json`.
|
|
- Truth correction: `approved != applied`.
|
|
- Applied canary: proposal `00957f6c-9883-4015-95a4-6b09367efb0e` and edge `c167933e-d513-4f43-9335-d5d8aeb259f2`.
|
|
- Staged write canary: proposal `8dfedb3f-3aa4-4200-970f-4c0016f6869f`, status `pending_review`, with no new public canonical rows after the test timestamp.
|
|
- Open-ended triage receipt:
|
|
`docs/reports/leo-working-state-20260709/telegram-live-open-ended-suite-current.md`,
|
|
where Leo inferred the likely failure mode from "agents not working / same
|
|
state as last night" and explained the proposed, approved, and applied split
|
|
without exact IDs.
|
|
- Old rich proposal packets: `14fa5ecc...`, `ac036c9d...`, and `a64df080...` are not to be silently production-applied.
|
|
- Guarded apply proof: both generic and real Helmer v3 receipts pass `37/37` in
|
|
disposable PostgreSQL; production Helmer remains unapplied.
|
|
- Clone-bound handler proof: the real VPS `GatewayRunner` can inspect the full
|
|
current-data disposable clone, discover the Helmer proposal, distinguish
|
|
`approved` from `applied`, and produce bound bridge-call evidence without a
|
|
Telegram post or production change. The proof surface must expose no delivery
|
|
adapters or send tool, restrict terminal execution to clone-bound read-only KB
|
|
verbs, kill the handler process group on timeout, and compare production
|
|
row-content fingerprints before and after. Treat open-ended latency and
|
|
retries as a separate reliability dimension; a correct answer after a failed
|
|
tool call is recovered behavior, not a clean pass, and one correct answer does
|
|
not establish a stable pass rate.
|
|
- Source-composition proof: the real VPS `GatewayRunner` passed `34/34` in a
|
|
fresh no-send full-data clone. It searched existing knowledge, extracted two
|
|
hash-bound conflicting claims from a new document and post, staged a strict
|
|
proposal, preserved an immutable separated approval, applied exact canonical
|
|
rows, reopened in a new child process, recalled the prior marker, discovered
|
|
the new proposal/claims without supplied IDs, read exact evidence/source UUIDs
|
|
and edges, and explained `approved != applied`. Production DB fingerprints,
|
|
service PID/restarts, and live bridge hashes stayed unchanged; all disposable
|
|
containers, volumes, profiles, credentials, and run directories were removed.
|
|
This is clone proof, not Telegram delivery or production apply proof.
|
|
|
|
## Direct Questions
|
|
|
|
For vague or no-context questions, answer directly and then name the one action
|
|
that would change the proof:
|
|
|
|
- "Did the DB change?": split applied rows, approved-but-unapplied proposals,
|
|
pending rows, and canceled rows. Say `Approved is not the same as applied`.
|
|
- "Is Helmer in Leo?": say `no, not canonical` until the exact claims, sources,
|
|
evidence, edges, reasoning tool, and applied ledger read back from production.
|
|
- "Did the decision matrix approve it?": verify matrix tables first; reviewer
|
|
approval is not a matrix vote.
|
|
- "Are document pointers the blocker?": answer `not just pointer mismatch` and
|
|
separate files, source refs, canonical source rows, review, and apply.
|
|
- "Can I demo KB mutation?": split staging-demo truth from canonical-apply truth.
|
|
- "Did editing SOUL.md change identity?": no canonical identity change without
|
|
row IDs plus render/sync postflight.
|
|
|
|
End no-context answers with exactly one `Next proof-changing follow-up:` line.
|
|
|
|
## Current Verdict - 2026-07-14
|
|
|
|
Use `not fully yet` for the whole m3taversal-standard question until every row below
|
|
is green at its required tier:
|
|
|
|
- VPS runtime, database-first direct questions, and intentional restart
|
|
survival are proven at no-send/runtime tier. The newest captured VPS canonical
|
|
DB has `39` tables, `52,167` rows, and `29` proposals.
|
|
- One natural ID-free VPS claim challenge now passes the complete bounded chain:
|
|
discover claim, inspect body/evidence/edges, challenge shallow support,
|
|
propose candidate claims, and preserve review before any write. This does not
|
|
erase the older broad 12-prompt failure; broad repeated reliability remains a
|
|
separate benchmark row.
|
|
- A fresh VPS snapshot was restored into a disposable private GCP Cloud SQL
|
|
database with exact `39/39`-table and `52,167/52,167`-row parity, zero catalog
|
|
mismatches, and unchanged production service state.
|
|
- The same GCP ID-free challenge passed `18/18` runtime checks and `6/6`
|
|
reasoning outcomes through the real Hermes runner. It used four successful
|
|
read-only, receipted calls and retrieved the expected claim plus both source
|
|
rows. No Telegram send or database write occurred.
|
|
- The GCP helper now gives deterministic receipts for every read, handles a
|
|
canonical-only restored DB without the optional audit schema, and returns an
|
|
honest zero-hit result instead of crashing.
|
|
- The reviewed helper/skill are merged and deployed. GCP survived a controlled
|
|
restart, then live `status` and `search` reached persistent Cloud SQL with
|
|
unchanged counts and stable receipts. Supported commands fail closed instead
|
|
of falling back to a different local database.
|
|
- The disposable GCP database and temporary profiles/processes were removed.
|
|
Persistent GCP `teleo_canonical` remains the older staging copy and has not
|
|
been promoted or cut over.
|
|
- Earlier Telegram-visible open-ended behavior and conversation memory are
|
|
retained proof, but there is no current Telegram-visible run of this full
|
|
challenge-to-candidate-claims flow.
|
|
- Canonical apply primitives are proven in isolation. No proposal from this
|
|
claim challenge was staged, reviewed, or production-applied, and broad
|
|
arbitrary document/post reconstruction from original sources remains open.
|
|
|
|
Do not answer `yes` merely because all repo tests pass. The final user-facing
|
|
proof is a visible Telegram conversation plus truthful canonical row readback.
|
|
|
|
## Hard m3taversal Question Bank
|
|
|
|
Use these without IDs, schema hints, or guardrail-heavy setup. The answer must
|
|
infer the relevant rows and end with one proof-changing follow-up:
|
|
|
|
1. `I approved Helmer weeks ago. Why isn't it real yet?`
|
|
2. `What can I show a partner in five minutes?`
|
|
3. `This PDF should update Leo. What happens now?`
|
|
4. `What changed in your identity since last week?`
|
|
5. `Make this claim canonical: <claim text>.`
|
|
6. `Are pending proposals stuck because sources are wrong?`
|
|
7. `What does our strategy depend on, and which evidence could overturn it?`
|
|
8. `What's the next KB change I should approve?`
|
|
|
|
Score the answer on directness, fresh canonical lookup, claim/evidence/source
|
|
reasoning, state semantics, uncertainty, row-level proof, and the usefulness of
|
|
the next action. Penalize asking m3taversal for IDs that Leo can discover itself.
|
|
|
|
Also fail the answer when it presents proposed architecture as current v1. The
|
|
current `public.claims` table has `text` and `superseded_by`, not `body`, generic
|
|
metadata, or forecast-resolution fields. Current `public.sources` has no
|
|
author/channel/date columns. Current accepted edge types are `supports`,
|
|
`challenges`, `requires`, `relates`, `contradicts`, `supersedes`,
|
|
`derives_from`, `cites`, `causes`, `constrains`, and `accelerates`.
|
|
|
|
## Required Answer Discipline
|
|
|
|
When Leo or a worker answers about KB state, it must say:
|
|
|
|
- What source was used: memory, Telegram history, `kb_stage`, canonical `public.*`, or filesystem/runtime.
|
|
- Which rows/tables changed.
|
|
- Which proposal IDs are only staged/approved.
|
|
- Which canonical rows exist.
|
|
- What remains pending or deferred.
|
|
|
|
## Not Done
|
|
|
|
Leo is not working just because:
|
|
|
|
- the systemd service is active,
|
|
- Bot API calls work,
|
|
- a local handler canary passed,
|
|
- a proposal was approved,
|
|
- a packet was generated,
|
|
- a clone rehearsal passed.
|
|
- `NRestarts=0` was observed without an intentional restart-cycle proof.
|
|
|
|
Those are evidence. The target is Telegram-visible behavior plus DB-state truth plus a guarded path to canonical rows.
|
|
|
|
A synthetic one-claim proposal proves only the guarded mutation primitive. It
|
|
does not prove KB composition. Composition requires a source that was absent at
|
|
baseline, model-driven extraction into linked source/evidence/claim proposals,
|
|
dedupe/conflict readback, guarded canonical apply, and an open-ended answer
|
|
grounded in the new rows after an isolated handler restart.
|
|
|
|
## Proof Files
|
|
|
|
Start with `docs/reports/leo-working-state-20260709/working-leo-current-proof-20260712.md`
|
|
and its JSON companion for the current whole-system verdict and completion rule.
|
|
Use `docs/reports/leo-working-state-20260709/current-truth-index.md` for current artifact paths.
|
|
Use `docs/reports/leo-working-state-20260709/leo-source-composition-clone-checkpoint-current.md`
|
|
and its JSON companion for the current source-composition claim ceiling.
|
|
For restart survival, use `scripts/collect_leo_restart_survival_proof.py --execute-restart`; expected artifacts are `docs/reports/leo-working-state-20260709/leo-restart-survival-proof-current.json` and `.md`.
|