teleo-infrastructure/.agents/skills/working-leo-m3taversal-outcomes/SKILL.md
twentyOne2x 15c30f1fbe
Some checks are pending
CI / lint-and-test (push) Waiting to run
Make Leo skill onboarding clean-context reproducible (#150)
* make Leo skill onboarding clean-context reproducible

* harden Leo clean-context skill acceptance
2026-07-15 03:19:15 +02:00

220 lines
12 KiB
Markdown

---
name: working-leo-m3taversal-outcomes
description: "Use when defining, testing, or repairing Leo against m3taversal's expected outcomes: Telegram memory, critical reasoning, canonical KB truth, proposed-vs-approved-vs-applied state, and guarded DB manipulation."
---
# Working Leo / m3taversal Outcomes
## Job
Keep Leo work anchored to what m3taversal appears to mean by "working": not just answers, but remembered context, truthful KB state, and approved concrete changes becoming canonical rows through a guarded proof path.
## Participant Identity Rule
- Address `@m3taversal` only as `m3taversal`, exactly.
- Never infer, invent, shorten, or substitute a personal name from a session
header, memory, identity document, another chat, or another participant.
- Resolve the speaker from the current Telegram update. Do not transfer identity
across users when someone replies, tags an account, or joins the thread.
- The legacy database value `m3ta` may be reported only as a quoted stored
reviewer handle. It is never a form of address.
- Standard response labels are neutral. Use `Next proof-changing follow-up:`.
## Trigger Phrases
- "working Leo"
- "m3taversal expected outcomes"
- "m3taversal says Leo is broken"
- "able to manipulate the knowledge base"
- "same state as last night"
- "critical reasoning behavior"
## Definition Of Working
A working Leo is a Telegram-facing agent that:
1. Remembers the current operator conversation.
2. Grounds KB answers in canonical Teleo Postgres rows when claiming KB truth.
3. Distinguishes `proposed`, `pending_review`, `approved`, `applied`, and `not applied`.
4. Does not say an approval changed canonical DB state when only `kb_stage` changed.
5. Can answer vague, high-level m3taversal-style incident prompts without being spoon-fed exact proposal IDs.
6. Can compose the KB from a previously unindexed document, URL, or tweet-like
source: retain a byte/hash-bound source locator, extract atomic claims and
exact evidence excerpts, preserve useful metadata, and link every claim to
its evidence and source.
7. Detects duplicates, conflicts, updates, and insufficient evidence before
staging; uncertainty must remain visible instead of being normalized into a
stronger claim.
8. Can stage those concrete KB changes from Telegram with enough structure for
human review, without making staged content canonical.
9. Can move approved concrete changes through a guarded apply path when authorized.
10. Retains m3taversal's caveats and review notes in source/evidence/proposal rows.
11. Reasons over claims, evidence, sources, edges, and open conflicts as a graph,
and can explain which rows support or weaken an answer without being given IDs.
12. Rebuilds any compiled identity/workspace artifact deterministically from
canonical DB rows, reports the source rows and freshness boundary, and does
not treat edits to a generated artifact as canonical knowledge.
13. Produces before/after table-level proof and service stability readback.
14. Survives an intentional `leoclean-gateway.service` restart: active before, active after, canonical KB counts unchanged, and a no-post handler smoke still answers.
15. For a reviewed graph bundle, produces exact payload-controlled row
projections, exact table deltas, source-byte binding, and cleanup proof in a
disposable runtime before any production permission or apply window.
## Current Benchmark Cases
- Historical marker-memory receipt:
`docs/reports/leo-working-state-20260709/telegram-live-canary-current.json`.
- Truth correction: `approved != applied`.
- Applied canary: proposal `00957f6c-9883-4015-95a4-6b09367efb0e` and edge `c167933e-d513-4f43-9335-d5d8aeb259f2`.
- Staged write canary: proposal `8dfedb3f-3aa4-4200-970f-4c0016f6869f`, status `pending_review`, with no new public canonical rows after the test timestamp.
- Open-ended triage receipt:
`docs/reports/leo-working-state-20260709/telegram-live-open-ended-suite-current.md`,
where Leo inferred the likely failure mode from "agents not working / same
state as last night" and explained the proposed, approved, and applied split
without exact IDs.
- Old rich proposal packets: `14fa5ecc...`, `ac036c9d...`, and `a64df080...` are not to be silently production-applied.
- Guarded apply proof: both generic and real Helmer v3 receipts pass `37/37` in
disposable PostgreSQL; production Helmer remains unapplied.
- Clone-bound handler proof: the real VPS `GatewayRunner` can inspect the full
current-data disposable clone, discover the Helmer proposal, distinguish
`approved` from `applied`, and produce bound bridge-call evidence without a
Telegram post or production change. The proof surface must expose no delivery
adapters or send tool, restrict terminal execution to clone-bound read-only KB
verbs, kill the handler process group on timeout, and compare production
row-content fingerprints before and after. Treat open-ended latency and
retries as a separate reliability dimension; a correct answer after a failed
tool call is recovered behavior, not a clean pass, and one correct answer does
not establish a stable pass rate.
- Source-composition proof: the real VPS `GatewayRunner` passed `34/34` in a
fresh no-send full-data clone. It searched existing knowledge, extracted two
hash-bound conflicting claims from a new document and post, staged a strict
proposal, preserved an immutable separated approval, applied exact canonical
rows, reopened in a new child process, recalled the prior marker, discovered
the new proposal/claims without supplied IDs, read exact evidence/source UUIDs
and edges, and explained `approved != applied`. Production DB fingerprints,
service PID/restarts, and live bridge hashes stayed unchanged; all disposable
containers, volumes, profiles, credentials, and run directories were removed.
This is clone proof, not Telegram delivery or production apply proof.
## Direct Questions
For vague or no-context questions, answer directly and then name the one action
that would change the proof:
- "Did the DB change?": split applied rows, approved-but-unapplied proposals,
pending rows, and canceled rows. Say `Approved is not the same as applied`.
- "Is Helmer in Leo?": say `no, not canonical` until the exact claims, sources,
evidence, edges, reasoning tool, and applied ledger read back from production.
- "Did the decision matrix approve it?": verify matrix tables first; reviewer
approval is not a matrix vote.
- "Are document pointers the blocker?": answer `not just pointer mismatch` and
separate files, source refs, canonical source rows, review, and apply.
- "Can I demo KB mutation?": split staging-demo truth from canonical-apply truth.
- "Did editing SOUL.md change identity?": no canonical identity change without
row IDs plus render/sync postflight.
End no-context answers with exactly one `Next proof-changing follow-up:` line.
## Current Verdict - 2026-07-14
Use `not fully yet` for the whole m3taversal-standard question until every row below
is green at its required tier:
- VPS runtime, database-first direct questions, and intentional restart
survival are proven at no-send/runtime tier. The newest captured VPS canonical
DB has `39` tables, `52,167` rows, and `29` proposals.
- One natural ID-free VPS claim challenge now passes the complete bounded chain:
discover claim, inspect body/evidence/edges, challenge shallow support,
propose candidate claims, and preserve review before any write. This does not
erase the older broad 12-prompt failure; broad repeated reliability remains a
separate benchmark row.
- A fresh VPS snapshot was restored into a disposable private GCP Cloud SQL
database with exact `39/39`-table and `52,167/52,167`-row parity, zero catalog
mismatches, and unchanged production service state.
- The same GCP ID-free challenge passed `18/18` runtime checks and `6/6`
reasoning outcomes through the real Hermes runner. It used four successful
read-only, receipted calls and retrieved the expected claim plus both source
rows. No Telegram send or database write occurred.
- The GCP helper now gives deterministic receipts for every read, handles a
canonical-only restored DB without the optional audit schema, and returns an
honest zero-hit result instead of crashing.
- The reviewed helper/skill are merged and deployed. GCP survived a controlled
restart, then live `status` and `search` reached persistent Cloud SQL with
unchanged counts and stable receipts. Supported commands fail closed instead
of falling back to a different local database.
- The disposable GCP database and temporary profiles/processes were removed.
Persistent GCP `teleo_canonical` remains the older staging copy and has not
been promoted or cut over.
- Earlier Telegram-visible open-ended behavior and conversation memory are
retained proof, but there is no current Telegram-visible run of this full
challenge-to-candidate-claims flow.
- Canonical apply primitives are proven in isolation. No proposal from this
claim challenge was staged, reviewed, or production-applied, and broad
arbitrary document/post reconstruction from original sources remains open.
Do not answer `yes` merely because all repo tests pass. The final user-facing
proof is a visible Telegram conversation plus truthful canonical row readback.
## Hard m3taversal Question Bank
Use these without IDs, schema hints, or guardrail-heavy setup. The answer must
infer the relevant rows and end with one proof-changing follow-up:
1. `I approved Helmer weeks ago. Why isn't it real yet?`
2. `What can I show a partner in five minutes?`
3. `This PDF should update Leo. What happens now?`
4. `What changed in your identity since last week?`
5. `Make this claim canonical: <claim text>.`
6. `Are pending proposals stuck because sources are wrong?`
7. `What does our strategy depend on, and which evidence could overturn it?`
8. `What's the next KB change I should approve?`
Score the answer on directness, fresh canonical lookup, claim/evidence/source
reasoning, state semantics, uncertainty, row-level proof, and the usefulness of
the next action. Penalize asking m3taversal for IDs that Leo can discover itself.
Also fail the answer when it presents proposed architecture as current v1. The
current `public.claims` table has `text` and `superseded_by`, not `body`, generic
metadata, or forecast-resolution fields. Current `public.sources` has no
author/channel/date columns. Current accepted edge types are `supports`,
`challenges`, `requires`, `relates`, `contradicts`, `supersedes`,
`derives_from`, `cites`, `causes`, `constrains`, and `accelerates`.
## Required Answer Discipline
When Leo or a worker answers about KB state, it must say:
- What source was used: memory, Telegram history, `kb_stage`, canonical `public.*`, or filesystem/runtime.
- Which rows/tables changed.
- Which proposal IDs are only staged/approved.
- Which canonical rows exist.
- What remains pending or deferred.
## Not Done
Leo is not working just because:
- the systemd service is active,
- Bot API calls work,
- a local handler canary passed,
- a proposal was approved,
- a packet was generated,
- a clone rehearsal passed.
- `NRestarts=0` was observed without an intentional restart-cycle proof.
Those are evidence. The target is Telegram-visible behavior plus DB-state truth plus a guarded path to canonical rows.
A synthetic one-claim proposal proves only the guarded mutation primitive. It
does not prove KB composition. Composition requires a source that was absent at
baseline, model-driven extraction into linked source/evidence/claim proposals,
dedupe/conflict readback, guarded canonical apply, and an open-ended answer
grounded in the new rows after an isolated handler restart.
## Proof Files
Start with `docs/reports/leo-working-state-20260709/working-leo-current-proof-20260712.md`
and its JSON companion for the current whole-system verdict and completion rule.
Use `docs/reports/leo-working-state-20260709/current-truth-index.md` for current artifact paths.
Use `docs/reports/leo-working-state-20260709/leo-source-composition-clone-checkpoint-current.md`
and its JSON companion for the current source-composition claim ceiling.
For restart survival, use `scripts/collect_leo_restart_survival_proof.py --execute-restart`; expected artifacts are `docs/reports/leo-working-state-20260709/leo-restart-survival-proof-current.json` and `.md`.