teleo-infrastructure/.agents/skills/working-leo-m3taversal-outcomes/SKILL.md
twentyOne2x 15c30f1fbe
Some checks are pending
CI / lint-and-test (push) Waiting to run
Make Leo skill onboarding clean-context reproducible (#150)
* make Leo skill onboarding clean-context reproducible

* harden Leo clean-context skill acceptance
2026-07-15 03:19:15 +02:00

12 KiB

name description
working-leo-m3taversal-outcomes Use when defining, testing, or repairing Leo against m3taversal's expected outcomes: Telegram memory, critical reasoning, canonical KB truth, proposed-vs-approved-vs-applied state, and guarded DB manipulation.

Working Leo / m3taversal Outcomes

Job

Keep Leo work anchored to what m3taversal appears to mean by "working": not just answers, but remembered context, truthful KB state, and approved concrete changes becoming canonical rows through a guarded proof path.

Participant Identity Rule

  • Address @m3taversal only as m3taversal, exactly.
  • Never infer, invent, shorten, or substitute a personal name from a session header, memory, identity document, another chat, or another participant.
  • Resolve the speaker from the current Telegram update. Do not transfer identity across users when someone replies, tags an account, or joins the thread.
  • The legacy database value m3ta may be reported only as a quoted stored reviewer handle. It is never a form of address.
  • Standard response labels are neutral. Use Next proof-changing follow-up:.

Trigger Phrases

  • "working Leo"
  • "m3taversal expected outcomes"
  • "m3taversal says Leo is broken"
  • "able to manipulate the knowledge base"
  • "same state as last night"
  • "critical reasoning behavior"

Definition Of Working

A working Leo is a Telegram-facing agent that:

  1. Remembers the current operator conversation.
  2. Grounds KB answers in canonical Teleo Postgres rows when claiming KB truth.
  3. Distinguishes proposed, pending_review, approved, applied, and not applied.
  4. Does not say an approval changed canonical DB state when only kb_stage changed.
  5. Can answer vague, high-level m3taversal-style incident prompts without being spoon-fed exact proposal IDs.
  6. Can compose the KB from a previously unindexed document, URL, or tweet-like source: retain a byte/hash-bound source locator, extract atomic claims and exact evidence excerpts, preserve useful metadata, and link every claim to its evidence and source.
  7. Detects duplicates, conflicts, updates, and insufficient evidence before staging; uncertainty must remain visible instead of being normalized into a stronger claim.
  8. Can stage those concrete KB changes from Telegram with enough structure for human review, without making staged content canonical.
  9. Can move approved concrete changes through a guarded apply path when authorized.
  10. Retains m3taversal's caveats and review notes in source/evidence/proposal rows.
  11. Reasons over claims, evidence, sources, edges, and open conflicts as a graph, and can explain which rows support or weaken an answer without being given IDs.
  12. Rebuilds any compiled identity/workspace artifact deterministically from canonical DB rows, reports the source rows and freshness boundary, and does not treat edits to a generated artifact as canonical knowledge.
  13. Produces before/after table-level proof and service stability readback.
  14. Survives an intentional leoclean-gateway.service restart: active before, active after, canonical KB counts unchanged, and a no-post handler smoke still answers.
  15. For a reviewed graph bundle, produces exact payload-controlled row projections, exact table deltas, source-byte binding, and cleanup proof in a disposable runtime before any production permission or apply window.

Current Benchmark Cases

  • Historical marker-memory receipt: docs/reports/leo-working-state-20260709/telegram-live-canary-current.json.
  • Truth correction: approved != applied.
  • Applied canary: proposal 00957f6c-9883-4015-95a4-6b09367efb0e and edge c167933e-d513-4f43-9335-d5d8aeb259f2.
  • Staged write canary: proposal 8dfedb3f-3aa4-4200-970f-4c0016f6869f, status pending_review, with no new public canonical rows after the test timestamp.
  • Open-ended triage receipt: docs/reports/leo-working-state-20260709/telegram-live-open-ended-suite-current.md, where Leo inferred the likely failure mode from "agents not working / same state as last night" and explained the proposed, approved, and applied split without exact IDs.
  • Old rich proposal packets: 14fa5ecc..., ac036c9d..., and a64df080... are not to be silently production-applied.
  • Guarded apply proof: both generic and real Helmer v3 receipts pass 37/37 in disposable PostgreSQL; production Helmer remains unapplied.
  • Clone-bound handler proof: the real VPS GatewayRunner can inspect the full current-data disposable clone, discover the Helmer proposal, distinguish approved from applied, and produce bound bridge-call evidence without a Telegram post or production change. The proof surface must expose no delivery adapters or send tool, restrict terminal execution to clone-bound read-only KB verbs, kill the handler process group on timeout, and compare production row-content fingerprints before and after. Treat open-ended latency and retries as a separate reliability dimension; a correct answer after a failed tool call is recovered behavior, not a clean pass, and one correct answer does not establish a stable pass rate.
  • Source-composition proof: the real VPS GatewayRunner passed 34/34 in a fresh no-send full-data clone. It searched existing knowledge, extracted two hash-bound conflicting claims from a new document and post, staged a strict proposal, preserved an immutable separated approval, applied exact canonical rows, reopened in a new child process, recalled the prior marker, discovered the new proposal/claims without supplied IDs, read exact evidence/source UUIDs and edges, and explained approved != applied. Production DB fingerprints, service PID/restarts, and live bridge hashes stayed unchanged; all disposable containers, volumes, profiles, credentials, and run directories were removed. This is clone proof, not Telegram delivery or production apply proof.

Direct Questions

For vague or no-context questions, answer directly and then name the one action that would change the proof:

  • "Did the DB change?": split applied rows, approved-but-unapplied proposals, pending rows, and canceled rows. Say Approved is not the same as applied.
  • "Is Helmer in Leo?": say no, not canonical until the exact claims, sources, evidence, edges, reasoning tool, and applied ledger read back from production.
  • "Did the decision matrix approve it?": verify matrix tables first; reviewer approval is not a matrix vote.
  • "Are document pointers the blocker?": answer not just pointer mismatch and separate files, source refs, canonical source rows, review, and apply.
  • "Can I demo KB mutation?": split staging-demo truth from canonical-apply truth.
  • "Did editing SOUL.md change identity?": no canonical identity change without row IDs plus render/sync postflight.

End no-context answers with exactly one Next proof-changing follow-up: line.

Current Verdict - 2026-07-14

Use not fully yet for the whole m3taversal-standard question until every row below is green at its required tier:

  • VPS runtime, database-first direct questions, and intentional restart survival are proven at no-send/runtime tier. The newest captured VPS canonical DB has 39 tables, 52,167 rows, and 29 proposals.
  • One natural ID-free VPS claim challenge now passes the complete bounded chain: discover claim, inspect body/evidence/edges, challenge shallow support, propose candidate claims, and preserve review before any write. This does not erase the older broad 12-prompt failure; broad repeated reliability remains a separate benchmark row.
  • A fresh VPS snapshot was restored into a disposable private GCP Cloud SQL database with exact 39/39-table and 52,167/52,167-row parity, zero catalog mismatches, and unchanged production service state.
  • The same GCP ID-free challenge passed 18/18 runtime checks and 6/6 reasoning outcomes through the real Hermes runner. It used four successful read-only, receipted calls and retrieved the expected claim plus both source rows. No Telegram send or database write occurred.
  • The GCP helper now gives deterministic receipts for every read, handles a canonical-only restored DB without the optional audit schema, and returns an honest zero-hit result instead of crashing.
  • The reviewed helper/skill are merged and deployed. GCP survived a controlled restart, then live status and search reached persistent Cloud SQL with unchanged counts and stable receipts. Supported commands fail closed instead of falling back to a different local database.
  • The disposable GCP database and temporary profiles/processes were removed. Persistent GCP teleo_canonical remains the older staging copy and has not been promoted or cut over.
  • Earlier Telegram-visible open-ended behavior and conversation memory are retained proof, but there is no current Telegram-visible run of this full challenge-to-candidate-claims flow.
  • Canonical apply primitives are proven in isolation. No proposal from this claim challenge was staged, reviewed, or production-applied, and broad arbitrary document/post reconstruction from original sources remains open.

Do not answer yes merely because all repo tests pass. The final user-facing proof is a visible Telegram conversation plus truthful canonical row readback.

Hard m3taversal Question Bank

Use these without IDs, schema hints, or guardrail-heavy setup. The answer must infer the relevant rows and end with one proof-changing follow-up:

  1. I approved Helmer weeks ago. Why isn't it real yet?
  2. What can I show a partner in five minutes?
  3. This PDF should update Leo. What happens now?
  4. What changed in your identity since last week?
  5. Make this claim canonical: <claim text>.
  6. Are pending proposals stuck because sources are wrong?
  7. What does our strategy depend on, and which evidence could overturn it?
  8. What's the next KB change I should approve?

Score the answer on directness, fresh canonical lookup, claim/evidence/source reasoning, state semantics, uncertainty, row-level proof, and the usefulness of the next action. Penalize asking m3taversal for IDs that Leo can discover itself.

Also fail the answer when it presents proposed architecture as current v1. The current public.claims table has text and superseded_by, not body, generic metadata, or forecast-resolution fields. Current public.sources has no author/channel/date columns. Current accepted edge types are supports, challenges, requires, relates, contradicts, supersedes, derives_from, cites, causes, constrains, and accelerates.

Required Answer Discipline

When Leo or a worker answers about KB state, it must say:

  • What source was used: memory, Telegram history, kb_stage, canonical public.*, or filesystem/runtime.
  • Which rows/tables changed.
  • Which proposal IDs are only staged/approved.
  • Which canonical rows exist.
  • What remains pending or deferred.

Not Done

Leo is not working just because:

  • the systemd service is active,
  • Bot API calls work,
  • a local handler canary passed,
  • a proposal was approved,
  • a packet was generated,
  • a clone rehearsal passed.
  • NRestarts=0 was observed without an intentional restart-cycle proof.

Those are evidence. The target is Telegram-visible behavior plus DB-state truth plus a guarded path to canonical rows.

A synthetic one-claim proposal proves only the guarded mutation primitive. It does not prove KB composition. Composition requires a source that was absent at baseline, model-driven extraction into linked source/evidence/claim proposals, dedupe/conflict readback, guarded canonical apply, and an open-ended answer grounded in the new rows after an isolated handler restart.

Proof Files

Start with docs/reports/leo-working-state-20260709/working-leo-current-proof-20260712.md and its JSON companion for the current whole-system verdict and completion rule. Use docs/reports/leo-working-state-20260709/current-truth-index.md for current artifact paths. Use docs/reports/leo-working-state-20260709/leo-source-composition-clone-checkpoint-current.md and its JSON companion for the current source-composition claim ceiling. For restart survival, use scripts/collect_leo_restart_survival_proof.py --execute-restart; expected artifacts are docs/reports/leo-working-state-20260709/leo-restart-survival-proof-current.json and .md.