diff --git a/.agents/skills/leo-telegram-canary-ops/SKILL.md b/.agents/skills/leo-telegram-canary-ops/SKILL.md index fe070d1..0788e1c 100644 --- a/.agents/skills/leo-telegram-canary-ops/SKILL.md +++ b/.agents/skills/leo-telegram-canary-ops/SKILL.md @@ -43,7 +43,7 @@ Name the canary before sending: - memory, - KB audit truth, - proposed-vs-applied truth, -- open-ended Cory-style triage, +- open-ended m3taversal-style triage, - staged write, - no-canonical-mutation proof, - screenshot/readback proof. @@ -58,7 +58,7 @@ Ask Leo for machine-checkable reply markers like: - `LIVE_TG_TURN3_STAGED` - `LIVE_TG_TURN4_READBACK` -Do not rely only on exact-ID prompts. At least one representative canary should be a vague operator-style prompt such as: Cory says the agents are not working, the KB is in the same state as last night, and Leo should be able to manipulate the KB; ask Leo what that means, what it would inspect first, what fixed means, and how it separates proposed, approved, and applied. Keep this canary read-only unless the task explicitly authorizes a staging or apply step. +Do not rely only on exact-ID prompts. At least one representative canary should be a vague operator-style prompt such as: m3taversal says the agents are not working, the KB is in the same state as last night, and Leo should be able to manipulate the KB; ask Leo what that means, what it would inspect first, what fixed means, and how it separates proposed, approved, and applied. Keep this canary read-only unless the task explicitly authorizes a staging or apply step. ## Required Proof @@ -77,6 +77,7 @@ After each live Telegram canary: - Live memory and KB audit passed for marker `WL-LIVE-TG-CORY-20260709`. - Live staged write passed for `WL-LIVE-TG-CORY-20260709-T3`. - Readback passed for `WL-LIVE-TG-CORY-20260709-T4`. -- Open-ended Cory-style read-only triage passed for `WL-LIVE-TG-CORY-OPEN-20260709`. +- Open-ended m3taversal-style read-only triage passed for the legacy marker + `WL-LIVE-TG-CORY-OPEN-20260709`. Use `docs/reports/leo-working-state-20260709/current-truth-index.md` for artifact paths. diff --git a/.agents/skills/teleo-gcp-parity-ops/SKILL.md b/.agents/skills/teleo-gcp-parity-ops/SKILL.md index 228f7c2..c295712 100644 --- a/.agents/skills/teleo-gcp-parity-ops/SKILL.md +++ b/.agents/skills/teleo-gcp-parity-ops/SKILL.md @@ -1,6 +1,6 @@ --- name: teleo-gcp-parity-ops -description: Use for passwordless Teleo GCP VM access, private Cloud SQL canonical parity, GCP Leo runtime readback, Cory no-send replay, rollback, and cleanup without collapsing VPS proof into GCP proof. +description: Use for passwordless Teleo GCP VM access, private Cloud SQL canonical parity, GCP Leo runtime readback, m3taversal no-send replay, rollback, and cleanup without collapsing VPS proof into GCP proof. --- # Teleo GCP Parity Ops @@ -73,7 +73,7 @@ Do not describe a passing Hermes memory sync as canonical KB parity. removed. `teleo_canonical_pre_20260712t1905z` is retained disabled, with zero connections, as the verified pre-swap rollback point. - A real no-send model replay returned six replies and nominally scored `6/6`, - but it is rejected as Cory-standard proof because `DC-03` and `DC-05` + but it is rejected as m3taversal-standard proof because `DC-03` and `DC-05` printed zero canonical counts. The real database has `1837/4145/4916/4670/26`. - The harness now enables clone-bound, default-read-only `teleo-kb status`, runs @@ -130,7 +130,7 @@ children, upload/run directories, and any temporary client are absent. - Do not restart the live GCP gateway for tool-file synchronization unless a separate restart window is explicitly requested. - Do not call control-plane inventory, memory sync, route readiness, or a - nominal scorer pass full Cory parity. + nominal scorer pass full m3taversal parity. The strongest accepted claim requires exact DB parity plus real no-send model replies with truthful counts and cleanup. It still does not prove Telegram diff --git a/.agents/skills/teleo-leo-onboarding/SKILL.md b/.agents/skills/teleo-leo-onboarding/SKILL.md index 98724d1..af975f2 100644 --- a/.agents/skills/teleo-leo-onboarding/SKILL.md +++ b/.agents/skills/teleo-leo-onboarding/SKILL.md @@ -22,7 +22,7 @@ Orient the worker before action. Build a current, proof-linked understanding of - Teleo is the knowledge/agent infrastructure layer behind Leo. - Leo is the operator-facing agent expected to answer in Telegram, remember operator context, reason from canonical KB state, stage concrete KB changes, and support approved changes becoming canonical DB rows with proof. -- The immediate July 9 issue is not generic bot liveness. It is Cory/m3taversal's expectation that approved KB changes move beyond proposal state when appropriate. +- The immediate July 9 issue is not generic bot liveness. It is m3taversal's expectation that approved KB changes move beyond proposal state when appropriate. - The VPS is the currently proven Telegram-visible Leo surface. - GCP is a separate lane. The live gateway, private Cloud SQL canonical rows, and exact VPS-to-GCP database parity are proven. Direct passwordless SSH was diff --git a/.agents/skills/teleo-vps-runtime-ops/SKILL.md b/.agents/skills/teleo-vps-runtime-ops/SKILL.md index fe0e5a1..3d45e5d 100644 --- a/.agents/skills/teleo-vps-runtime-ops/SKILL.md +++ b/.agents/skills/teleo-vps-runtime-ops/SKILL.md @@ -121,7 +121,7 @@ sudo -u teleo HOME=/home/teleo \ --container \ --db teleo \ --prompt-id \ - --prompt "" \ + --prompt "" \ --expected-state approved \ --copy-model-auth \ --output /home/teleo/leo-checkpoint-reports/leo-clone-bound-checkpoint.json diff --git a/.agents/skills/working-leo-cory-outcomes/SKILL.md b/.agents/skills/working-leo-m3taversal-outcomes/SKILL.md similarity index 75% rename from .agents/skills/working-leo-cory-outcomes/SKILL.md rename to .agents/skills/working-leo-m3taversal-outcomes/SKILL.md index 7d49e2e..2963c80 100644 --- a/.agents/skills/working-leo-cory-outcomes/SKILL.md +++ b/.agents/skills/working-leo-m3taversal-outcomes/SKILL.md @@ -1,18 +1,29 @@ --- -name: working-leo-cory-outcomes -description: "Use when defining, testing, or repairing Leo against Cory/m3taversal's expected outcomes: Telegram memory, critical reasoning, canonical KB truth, proposed-vs-approved-vs-applied state, and guarded DB manipulation." +name: working-leo-m3taversal-outcomes +description: "Use when defining, testing, or repairing Leo against m3taversal's expected outcomes: Telegram memory, critical reasoning, canonical KB truth, proposed-vs-approved-vs-applied state, and guarded DB manipulation." --- -# Working Leo / Cory Outcomes +# Working Leo / m3taversal Outcomes ## Job -Keep Leo work anchored to what Cory/m3taversal appears to mean by "working": not just answers, but remembered context, truthful KB state, and approved concrete changes becoming canonical rows through a guarded proof path. +Keep Leo work anchored to what m3taversal appears to mean by "working": not just answers, but remembered context, truthful KB state, and approved concrete changes becoming canonical rows through a guarded proof path. + +## Participant Identity Rule + +- Address `@m3taversal` only as `m3taversal`, exactly. +- Never infer, invent, shorten, or substitute a personal name from a session + header, memory, identity document, another chat, or another participant. +- Resolve the speaker from the current Telegram update. Do not transfer identity + across users when someone replies, tags an account, or joins the thread. +- The legacy database value `m3ta` may be reported only as a quoted stored + reviewer handle. It is never a form of address. +- Standard response labels are neutral. Use `Next proof-changing follow-up:`. ## Trigger Phrases - "working Leo" -- "Cory expected outcomes" +- "m3taversal expected outcomes" - "m3taversal says Leo is broken" - "able to manipulate the knowledge base" - "same state as last night" @@ -26,7 +37,7 @@ A working Leo is a Telegram-facing agent that: 2. Grounds KB answers in canonical Teleo Postgres rows when claiming KB truth. 3. Distinguishes `proposed`, `pending_review`, `approved`, `applied`, and `not applied`. 4. Does not say an approval changed canonical DB state when only `kb_stage` changed. -5. Can answer vague, high-level Cory-style incident prompts without being spoon-fed exact proposal IDs. +5. Can answer vague, high-level m3taversal-style incident prompts without being spoon-fed exact proposal IDs. 6. Can compose the KB from a previously unindexed document, URL, or tweet-like source: retain a byte/hash-bound source locator, extract atomic claims and exact evidence excerpts, preserve useful metadata, and link every claim to @@ -37,7 +48,7 @@ A working Leo is a Telegram-facing agent that: 8. Can stage those concrete KB changes from Telegram with enough structure for human review, without making staged content canonical. 9. Can move approved concrete changes through a guarded apply path when authorized. -10. Retains Cory's caveats and review notes in source/evidence/proposal rows. +10. Retains m3taversal's caveats and review notes in source/evidence/proposal rows. 11. Reasons over claims, evidence, sources, edges, and open conflicts as a graph, and can explain which rows support or weaken an answer without being given IDs. 12. Rebuilds any compiled identity/workspace artifact deterministically from @@ -80,7 +91,7 @@ A working Leo is a Telegram-facing agent that: containers, volumes, profiles, credentials, and run directories were removed. This is clone proof, not Telegram delivery or production apply proof. -## Cory Direct Questions +## Direct Questions For vague or no-context questions, answer directly and then name the one action that would change the proof: @@ -97,11 +108,11 @@ that would change the proof: - "Did editing SOUL.md change identity?": no canonical identity change without row IDs plus render/sync postflight. -End no-context answers with exactly one `Next Cory-style follow-up:` line. +End no-context answers with exactly one `Next proof-changing follow-up:` line. -## Current Verdict - 2026-07-12 +## Current Verdict - 2026-07-13 -Use `not fully yet` for the whole Cory-standard question until every row below +Use `not fully yet` for the whole m3taversal-standard question until every row below is green at its required tier: - VPS runtime and restart survival: proven. @@ -109,7 +120,8 @@ is green at its required tier: strict-score `6/6` (`18/18` replies) with unchanged DB/service state and complete temporary-profile cleanup. - Telegram-visible open-ended read-only behavior and conversation memory: - proven by the earlier `OE-01` through `OE-05` suite. + proven for the earlier `OE-01` through `OE-05` suite, but that narrower suite + does not establish current broad reliability. - Telegram-visible no-context direct-claim suite: all six prompts/replies are captured with unchanged DB/service state, but the hardened score is `5/6`. `DC-05` proposes an incompatible legacy `add_edge` apply target. @@ -118,15 +130,22 @@ is green at its required tier: - Source composition: deterministic fixture proof plus a full-data, no-send, current-VPS clone checkpoint are proven. Arbitrary production document/tweet ingestion is not proven. -- GCP canonical DB parity and six DB-read routes: proven. Real model replay must - also pass exact count consistency; a nominal scorer pass with invented counts - is a failure. The current nominal `6/6` run is therefore rejected, and the - hardened rerun/clone cleanup waits on Google password reauthentication. +- Broad blind VPS behavior: operationally clean but semantically failed. + Twelve of twelve prompts returned, DB/service state stayed unchanged, and the + temporary profile was removed; independent strict judges accepted only + `1/12` and `2/12` outright. Failures include invented current schema fields + and edge types, handler proof described as Telegram-live, incorrect runtime + memory boundaries, and temporary memory treated as source provenance. +- GCP canonical DB parity and six DB-read routes: proven. A later adapter-free + model replay passed `6/6` with exact count consistency and unchanged state. + Durable on-demand operator access and cleanup of the retained replay clone + remain open because the expected operator service accounts are absent and the + privileged Google account requires human password/MFA reauthentication. Do not answer `yes` merely because all repo tests pass. The final user-facing proof is a visible Telegram conversation plus truthful canonical row readback. -## Hard Cory Question Bank +## Hard m3taversal Question Bank Use these without IDs, schema hints, or guardrail-heavy setup. The answer must infer the relevant rows and end with one proof-changing follow-up: @@ -142,7 +161,14 @@ infer the relevant rows and end with one proof-changing follow-up: Score the answer on directness, fresh canonical lookup, claim/evidence/source reasoning, state semantics, uncertainty, row-level proof, and the usefulness of -the next action. Penalize asking Cory for IDs that Leo can discover itself. +the next action. Penalize asking m3taversal for IDs that Leo can discover itself. + +Also fail the answer when it presents proposed architecture as current v1. The +current `public.claims` table has `text` and `superseded_by`, not `body`, generic +metadata, or forecast-resolution fields. Current `public.sources` has no +author/channel/date columns. Current accepted edge types are `supports`, +`challenges`, `requires`, `relates`, `contradicts`, `supersedes`, +`derives_from`, `cites`, `causes`, `constrains`, and `accelerates`. ## Required Answer Discipline diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 29c02e8..5bdd059 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -55,15 +55,17 @@ jobs: telegram/approvals.py \ scripts/check_crabbox_ci_contract.py \ scripts/check_llm_refinement_contract.py \ - scripts/build_working_leo_cory_outcome_sandbox.py \ + scripts/build_working_leo_m3taversal_outcome_sandbox.py \ scripts/replay_decision_engine_eval.py \ scripts/prove_phase1b_local.py \ scripts/run_gcp_generated_db_direct_claim_suite.py \ + scripts/run_leo_m3taversal_oos_handler_suite.py \ scripts/run_leo_clone_bound_handler_checkpoint.py \ + scripts/working_leo_m3taversal_oos_benchmark.py \ scripts/working_leo_open_ended_benchmark.py \ tests/test_agent_routing.py \ tests/test_assemble_telegram_visible_direct_claim_capture_receipt.py \ - tests/test_build_working_leo_cory_outcome_sandbox.py \ + tests/test_build_working_leo_m3taversal_outcome_sandbox.py \ tests/test_decision_engine_replay.py \ tests/test_evaluate_agent_routing.py \ tests/test_gcp_artifact_workflow.py \ @@ -79,6 +81,7 @@ jobs: tests/test_gcp_readiness_workflow.py \ tests/test_gcp_generated_db_direct_claim_suite.py \ tests/test_verify_postgres_parity_manifest.py \ + tests/test_working_leo_m3taversal_oos_benchmark.py \ tests/test_working_leo_open_ended_benchmark.py \ tests/test_phase1b_end_to_end.py \ tests/test_sqlite_to_postgres_dump.py \ diff --git a/docs/reports/leo-working-state-20260709/fable-leo-teleo-onboarding.md b/docs/reports/leo-working-state-20260709/fable-leo-teleo-onboarding.md index 2974240..7dbc69a 100644 --- a/docs/reports/leo-working-state-20260709/fable-leo-teleo-onboarding.md +++ b/docs/reports/leo-working-state-20260709/fable-leo-teleo-onboarding.md @@ -9,7 +9,7 @@ Use the Teleo repo skill pack at: Load these draft skills first: 1. `skills/teleo-leo-onboarding/SKILL.md` -2. `skills/working-leo-cory-outcomes/SKILL.md` +2. `skills/working-leo-m3taversal-outcomes/SKILL.md` 3. `skills/teleo-kb-db-change-workflow/SKILL.md` 4. `skills/teleo-vps-runtime-ops/SKILL.md` 5. `skills/teleo-proof-handoff/SKILL.md` diff --git a/docs/reports/leo-working-state-20260709/skill-pack-manifest.json b/docs/reports/leo-working-state-20260709/skill-pack-manifest.json index 8735a3c..613a99e 100644 --- a/docs/reports/leo-working-state-20260709/skill-pack-manifest.json +++ b/docs/reports/leo-working-state-20260709/skill-pack-manifest.json @@ -1,13 +1,13 @@ { "pack": "leo-teleo-skill-pack", "created_date": "2026-07-09", - "last_updated_utc": "2026-07-12T20:50:00Z", + "last_updated_utc": "2026-07-13T06:45:00Z", "status": "repo_native_validated", "contains_secrets": false, "production_mutation_authorized": false, "repo_skill_root": ".agents/skills", "optional_local_install_target": "/Users/user/.codex/skills", - "claim_ceiling": "Repo-native onboarding and operations skills. Generic and Helmer guarded-apply canaries pass 37/37 in disposable PostgreSQL. PR #72 source auto-synchronized to the VPS checkout without a gateway restart or profile-source delta; the permission migration was not applied, the apply worker remained disabled/inactive, and Helmer remained unapplied. Current source-composition/apply lifecycle proof is stronger and remains isolated. GCP gateway and exact canonical DB parity are proven across 39 tables and 52,164 rows. A nominal 6/6 GCP model replay was rejected for false printed counts; hardened replay and cleanup wait on Google password reauthentication because direct SSH no longer matches its firewall /32 and the dedicated OIDC/IAP provider is not bootstrapped.", + "claim_ceiling": "Repo-native onboarding and operations skills. Generic and Helmer guarded-apply canaries pass 37/37 in disposable PostgreSQL. PR #72 source auto-synchronized to the VPS checkout without a gateway restart or profile-source delta; the permission migration was not applied, the apply worker remained disabled/inactive, and Helmer remained unapplied. Current source-composition/apply lifecycle proof is stronger and remains isolated. GCP gateway and exact canonical DB parity are proven across 39 tables and 52,164 rows, and the hardened adapter-free model replay passes 6/6 with exact count consistency. Durable GCP operator identity and retained clone cleanup remain open. Broad blind VPS reasoning is not reliable yet: independent strict judges accepted only 1/12 and 2/12 outright. Current skills require exact m3taversal naming, neutral follow-up labels, and current-v1 schema truth.", "skills": [ { "name": "teleo-leo-onboarding", @@ -15,9 +15,9 @@ "path": ".agents/skills/teleo-leo-onboarding/SKILL.md" }, { - "name": "working-leo-cory-outcomes", - "role": "Definition of working Leo from Cory/m3taversal outcomes and tests", - "path": ".agents/skills/working-leo-cory-outcomes/SKILL.md" + "name": "working-leo-m3taversal-outcomes", + "role": "Definition of working Leo from m3taversal outcomes and tests", + "path": ".agents/skills/working-leo-m3taversal-outcomes/SKILL.md" }, { "name": "teleo-vps-runtime-ops", @@ -26,7 +26,7 @@ }, { "name": "teleo-gcp-parity-ops", - "role": "GCP operator access, private Cloud SQL parity, Cory replay, rollback, and cleanup", + "role": "GCP operator access, private Cloud SQL parity, m3taversal replay, rollback, and cleanup", "path": ".agents/skills/teleo-gcp-parity-ops/SKILL.md" }, { diff --git a/docs/reports/leo-working-state-20260709/skill-pack-readme.md b/docs/reports/leo-working-state-20260709/skill-pack-readme.md index 46eb8e4..6c8f299 100644 --- a/docs/reports/leo-working-state-20260709/skill-pack-readme.md +++ b/docs/reports/leo-working-state-20260709/skill-pack-readme.md @@ -7,7 +7,7 @@ This is the repo-native skill set under `.agents/skills/`. It is evidence-linked ## Skills In This Pack 1. `teleo-leo-onboarding`: company/product/architecture orientation before touching Leo or Teleo. -2. `working-leo-cory-outcomes`: Cory/m3taversal's expected Leo behavior and the current benchmark. +2. `working-leo-m3taversal-outcomes`: m3taversal's expected Leo behavior and the current benchmark. 3. `teleo-vps-runtime-ops`: VPS service, paths, Postgres, Docker, report sync, and stability checks. 4. `teleo-gcp-parity-ops`: passwordless GCP access, Cloud SQL parity, no-send replay, rollback, and cleanup. 5. `teleo-kb-db-change-workflow`: source composition plus approved proposal to canonical-row workflow with clone rehearsal and rollback. @@ -20,8 +20,8 @@ This is the repo-native skill set under `.agents/skills/`. It is evidence-linked - VPS Leo Telegram memory, KB audit, and staged write canaries are live-proven. - A strict canonical `add_edge` apply canary is live-proven. - Deterministic source composition, full-data clone composition/reasoning, and guarded approved-bundle application are isolated-proven with exact source/evidence/claim links, row deltas, rollback, and cleanup. Production rich packets remain unapplied. -- The live VPS runtime behavior should not be changed while the DB apply path is stabilized. -- GCP gateway liveness, private Cloud SQL identity, and exact VPS-to-GCP canonical parity are live-proven. The database matches across `39` tables and `52,164` rows. Direct passwordless SSH was proven but is currently unavailable because its firewall `/32` no longer matches the Mac's egress; the dedicated OIDC/IAP route is merged but not bootstrapped. Telegram delivery, GCP canonical mutation, ongoing replication, and cutover remain separate proof rows. +- Narrow VPS DB truth and restart behavior are proven. Broad blind reasoning is not yet reliable; exact participant naming, neutral labels, and current-v1 schema guards are now required. +- GCP gateway liveness, private Cloud SQL identity, exact VPS-to-GCP canonical parity, and hardened adapter-free model replay are live-proven. The database matches across `39` tables and `52,164` rows. Durable operator identity and retained replay-clone cleanup remain open because the intended target service accounts are absent. ## Key References @@ -31,7 +31,9 @@ This is the repo-native skill set under `.agents/skills/`. It is evidence-linked - Fable onboarding prompt: `docs/reports/leo-working-state-20260709/fable-leo-teleo-onboarding.md` - PR #72 VPS auto-deploy/runtime readback: `docs/reports/leo-working-state-20260709/pr72-vps-auto-deploy-runtime-nonchange-current.md` - GCP canonical parity: `docs/reports/leo-working-state-20260709/gcp-canonical-parity-live-20260712.json` -- GCP Cory replay: `docs/reports/leo-working-state-20260709/gcp-cory-model-replay-current.json` +- GCP model replay (legacy artifact filename): `docs/reports/leo-working-state-20260709/gcp-cory-model-replay-current.json` +- Three-day delivery delta: `docs/reports/leo-working-state-20260709/working-leo-three-day-delivery-summary-20260713.md` +- Blind out-of-sample audit: `docs/reports/leo-working-state-20260709/telegram-handler-blind-oos-audit-current.json` - GCP operator access gate: `docs/reports/leo-working-state-20260709/gcp-operator-access-blocker-current.json` - Skill manifest: `skill-pack-manifest.json` diff --git a/hermes-agent/leoclean-skills/gcp/teleo-kb-bridge/SKILL.md b/hermes-agent/leoclean-skills/gcp/teleo-kb-bridge/SKILL.md index 52b2c77..d93d162 100644 --- a/hermes-agent/leoclean-skills/gcp/teleo-kb-bridge/SKILL.md +++ b/hermes-agent/leoclean-skills/gcp/teleo-kb-bridge/SKILL.md @@ -72,17 +72,17 @@ If `search-proposals` finds an `approved` proposal with `applied_at` empty, say it is approved/staged or packet-ready but not canonical. Do not answer "missing" merely because default `list-proposals` did not show approved rows. -For these no-context direct claims, use this compact answer shape so Cory gets -the expected follow-up without needing to ask twice: +For these no-context direct claims, use this compact answer shape so the +operator gets the expected follow-up without needing to ask twice: 1. Direct answer: yes/no/partly, with the truth ceiling. 2. Readback used: the exact bridge commands or row facts checked. 3. Canonical vs staged split: name `public.*`, `kb_stage.kb_proposals`, status, proposal id, and `applied_at` when relevant. -4. Next Cory-style follow-up: the one proof-changing or admin action that would +4. Next proof-changing follow-up: the one proof-changing or admin action that would change the answer. -Always include the final line label `Next Cory-style follow-up:` for a +Always include the final line label `Next proof-changing follow-up:` for a no-context direct-claim answer. Do not omit it just because the answer seems complete. @@ -94,11 +94,42 @@ Use explicit no-overclaim wording when the canonical DB did not change: "I cannot claim canonical DB changed until `public.*` readback plus `applied_at`/postflight proof says it changed." -### Cory Direct-Claim Answer Contract +### Telegram Participant Naming Contract -For Cory-style no-context questions, keep the answer direct but include the +- Address `@m3taversal` only as `m3taversal`, exactly. Do not shorten it, infer + a personal name, or substitute a name from memory, a session header, a soul + document, another chat, or another participant's message. +- Resolve the current speaker from the current Telegram update. Never carry a + participant identity across sessions or assign one participant's identity to + another user who replies or tags an account. +- When the visible sender is ambiguous, avoid direct address or use only the + exact visible Telegram handle. Ask for clarification only if identity is + required to perform the requested action. +- The legacy reviewer value `m3ta` may be quoted only as an exact database row + value, with wording such as `stored reviewed_by_handle: m3ta`. It is not a + form of address and does not authorize a nickname. +- Keep answer labels neutral. Never put a participant's name in a standard + follow-up label. + +### Operator Direct-Claim Answer Contract + +For m3taversal-style no-context questions, keep the answer direct but include the proof language below. These are behavioral examples, not feature changes. +Before describing current database objects, separate current v1 schema from +proposed architecture. Current `public.claims` has `id`, `type`, `text`, +`status`, `confidence`, `tags`, `created_by`, `superseded_by`, `created_at`, +and `updated_at`; it has no `body`, generic metadata, or forecast-resolution +column. Current `public.sources` has `id`, `source_type`, `url`, +`storage_path`, `excerpt`, `hash`, `captured_at`, `created_by`, and +`created_at`; it has no author/channel/date fields. Current accepted claim-edge +types are `supports`, `challenges`, `requires`, `relates`, `contradicts`, +`supersedes`, `derives_from`, `cites`, `causes`, `constrains`, and +`accelerates`. Do not present a proposed v3 field, table, edge type, or policy +as shipped. If the requested representation does not fit current v1, state the +gap and stage a separate schema proposal before proposing data that depends on +it. + - "Did we actually update the KB?": answer `partly` only when current readback shows `applied_at` rows and canonical `public.*` rows. Otherwise say `mostly still proposals`; list applied, approved-but-not-applied, pending, @@ -179,7 +210,7 @@ template. Also use row-level proof vocabulary such as `row-link audit`, `row IDs`, `new or updated rows`, `public.*`, and `postflight proof` where relevant. End with exactly one final line beginning -`Next Cory-style follow-up:` that asks for or offers the next proof-changing +`Next proof-changing follow-up:` that asks for or offers the next proof-changing action. ## Telegram Rendering diff --git a/hermes-agent/leoclean-skills/vps/teleo-kb-bridge/SKILL.md b/hermes-agent/leoclean-skills/vps/teleo-kb-bridge/SKILL.md index 4ddd4cc..ac35c80 100644 --- a/hermes-agent/leoclean-skills/vps/teleo-kb-bridge/SKILL.md +++ b/hermes-agent/leoclean-skills/vps/teleo-kb-bridge/SKILL.md @@ -65,17 +65,17 @@ If `search-proposals` finds an `approved` proposal with `applied_at` empty, say it is approved/staged or packet-ready but not canonical. Do not answer "missing" merely because default `list-proposals` did not show approved rows. -For these no-context direct claims, use this compact answer shape so Cory gets -the expected follow-up without needing to ask twice: +For these no-context direct claims, use this compact answer shape so the +operator gets the expected follow-up without needing to ask twice: 1. Direct answer: yes/no/partly, with the truth ceiling. 2. Readback used: the exact bridge commands or row facts checked. 3. Canonical vs staged split: name `public.*`, `kb_stage.kb_proposals`, status, proposal id, and `applied_at` when relevant. -4. Next Cory-style follow-up: the one proof-changing or admin action that would +4. Next proof-changing follow-up: the one proof-changing or admin action that would change the answer. -Always include the final line label `Next Cory-style follow-up:` for a +Always include the final line label `Next proof-changing follow-up:` for a no-context direct-claim answer. Do not omit it just because the answer seems complete. @@ -87,11 +87,42 @@ Use explicit no-overclaim wording when the canonical DB did not change: "I cannot claim canonical DB changed until `public.*` readback plus `applied_at`/postflight proof says it changed." -### Cory Direct-Claim Answer Contract +### Telegram Participant Naming Contract -For Cory-style no-context questions, keep the answer direct but include the +- Address `@m3taversal` only as `m3taversal`, exactly. Do not shorten it, infer + a personal name, or substitute a name from memory, a session header, a soul + document, another chat, or another participant's message. +- Resolve the current speaker from the current Telegram update. Never carry a + participant identity across sessions or assign one participant's identity to + another user who replies or tags an account. +- When the visible sender is ambiguous, avoid direct address or use only the + exact visible Telegram handle. Ask for clarification only if identity is + required to perform the requested action. +- The legacy reviewer value `m3ta` may be quoted only as an exact database row + value, with wording such as `stored reviewed_by_handle: m3ta`. It is not a + form of address and does not authorize a nickname. +- Keep answer labels neutral. Never put a participant's name in a standard + follow-up label. + +### Operator Direct-Claim Answer Contract + +For m3taversal-style no-context questions, keep the answer direct but include the proof language below. These are behavioral examples, not feature changes. +Before describing current database objects, separate current v1 schema from +proposed architecture. Current `public.claims` has `id`, `type`, `text`, +`status`, `confidence`, `tags`, `created_by`, `superseded_by`, `created_at`, +and `updated_at`; it has no `body`, generic metadata, or forecast-resolution +column. Current `public.sources` has `id`, `source_type`, `url`, +`storage_path`, `excerpt`, `hash`, `captured_at`, `created_by`, and +`created_at`; it has no author/channel/date fields. Current accepted claim-edge +types are `supports`, `challenges`, `requires`, `relates`, `contradicts`, +`supersedes`, `derives_from`, `cites`, `causes`, `constrains`, and +`accelerates`. Do not present a proposed v3 field, table, edge type, or policy +as shipped. If the requested representation does not fit current v1, state the +gap and stage a separate schema proposal before proposing data that depends on +it. + - "Did we actually update the KB?": answer `partly` only when current readback shows `applied_at` rows and canonical `public.*` rows. Otherwise say `mostly still proposals`; list applied, approved-but-not-applied, pending, @@ -185,7 +216,7 @@ bounded read-only bridge command needed to fill it. Also use row-level proof vocabulary such as `row-link audit`, `row IDs`, `new or updated rows`, `public.*`, and `postflight proof` where relevant. End with exactly one final line beginning -`Next Cory-style follow-up:` that asks for or offers the next proof-changing +`Next proof-changing follow-up:` that asks for or offers the next proof-changing action. ## Telegram Rendering diff --git a/scripts/build_telegram_visible_direct_claim_authorization_packet.py b/scripts/build_telegram_visible_direct_claim_authorization_packet.py index bdc81df..126d234 100644 --- a/scripts/build_telegram_visible_direct_claim_authorization_packet.py +++ b/scripts/build_telegram_visible_direct_claim_authorization_packet.py @@ -63,7 +63,7 @@ def direct_claim_messages() -> list[dict[str, Any]]: "expected_follow_up": prompt["expected_follow_up"], "required_signals": list(prompt["required_signals"]), } - for prompt in benchmark.CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS + for prompt in benchmark.M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS ] @@ -141,7 +141,7 @@ def build_packet( "handler_direct_claim_score_passed": _score_passes(handler_score, expected_count=len(messages)), "all_evidence_paths_exist": all(item["exists"] and item["bytes"] > 0 for item in evidence_paths.values()), "raw_dc_messages_preserved": [item["message"] for item in messages] - == [prompt["message"] for prompt in benchmark.CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS], + == [prompt["message"] for prompt in benchmark.M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS], "telegram_visible_messages_not_sent_by_packet": True, "production_apply_not_authorized_by_packet": True, "gcp_parity_not_required_for_vps_telegram_test": gcp_probe.get("ready_for_gcp_readonly_parity") is not True, diff --git a/scripts/build_working_leo_cory_outcome_sandbox.py b/scripts/build_working_leo_m3taversal_outcome_sandbox.py similarity index 92% rename from scripts/build_working_leo_cory_outcome_sandbox.py rename to scripts/build_working_leo_m3taversal_outcome_sandbox.py index 32f1706..aaa33f5 100644 --- a/scripts/build_working_leo_cory_outcome_sandbox.py +++ b/scripts/build_working_leo_m3taversal_outcome_sandbox.py @@ -1,16 +1,16 @@ #!/usr/bin/env python3 -"""Build retained sandbox-first Cory outcome benchmark results. +"""Build retained sandbox-first m3taversal outcome benchmark results. The output intentionally mixes two evidence tiers: - OE-01..OE-05 are copied from retained live Telegram read-only evidence. - CS-01..CS-09 are sandbox fixture replies that encode the expected Working Leo - behavior for higher-risk Cory-style outcome prompts. -- DC-01..DC-06 are sandbox fixture replies for no-context Cory-style direct + behavior for higher-risk m3taversal-style outcome prompts. +- DC-01..DC-06 are sandbox fixture replies for no-context m3taversal-style direct claim questions with the expected follow-up action. This script does not call Leo, does not send Telegram messages, and does not -mutate the KB. Its job is to make the broad Cory-style benchmark executable and +mutate the KB. Its job is to make the broad m3taversal-style benchmark executable and scorable before any future disposable clone or live run attempts those prompts. """ @@ -26,7 +26,7 @@ import working_leo_open_ended_benchmark as benchmark REPO_REPORT_DIR = Path("docs/reports/leo-working-state-20260709") DEFAULT_LIVE_SUITE_SCORE = REPO_REPORT_DIR / "telegram-live-open-ended-suite-score-current.json" -DEFAULT_OUT = REPO_REPORT_DIR / "working-leo-cory-outcome-sandbox-results-current.json" +DEFAULT_OUT = REPO_REPORT_DIR / "working-leo-m3taversal-outcome-sandbox-results-current.json" CS_REPLIES: dict[str, str] = { @@ -74,7 +74,7 @@ CS_REPLIES: dict[str, str] = { ), "CS-06": ( "Demo answer: Leo works for Telegram-visible memory, KB audit, and guarded staging; strict existing-ID apply " - "has row-level proof. Proposed, approved, applied, and not applied remain separate. The rich Cory-approved " + "has row-level proof. Proposed, approved, applied, and not applied remain separate. The rich m3taversal-approved " "packet set is clone-proven and apply-readiness verified, but it is not production-applied. I would show the " "live Telegram proof, the canonical public.* readback, the apply-readiness artifact, and the service " "MainPID/NRestarts readback. The next smallest proof that changes the answer is explicit production " @@ -114,7 +114,7 @@ DC_REPLIES: dict[str, str] = { "applied are different states. The canonical answer comes from public.claims, public.sources, " "public.claim_edges, and public.claim_evidence, plus kb_stage.kb_proposals applied_at/readback. I need " "row-level before/after counts, row ids, and postflight proof; I cannot claim a DB update without proof. " - "Follow-up: I would ask which proposal or time window Cory means, then query kb_stage and public.* and " + "Follow-up: I would ask which proposal or time window m3taversal means, then query kb_stage and public.* and " "either show the pending gap or prepare the next guarded apply packet.\nDB readback: Proposal: " "`f004bbb2-ac9a-481f-b7b8-74319373ba6a`; Status: `applied`; Applied at: `2026-07-05`." ), @@ -123,7 +123,7 @@ DC_REPLIES: dict[str, str] = { "check public.reasoning_tools, public.claims, public.sources, public.claim_edges, public.claim_evidence, " "and the proposal ledger with postflight row counts before saying it is in Leo now. If production_apply " "is false, I cannot claim it is production-applied. Follow-up: I would show the readiness packet, ask for " - "explicit authorization if Cory wants it applied, run the integrated order, then postflight and Telegram " + "explicit authorization if m3taversal wants it applied, run the integrated order, then postflight and Telegram " "regression.\nDB readback: Proposal: `a64df080-8502-42e2-98f4-9bbdecb8da73`; Status: `approved`; " "Applied at: `none`." ), @@ -149,7 +149,7 @@ DC_REPLIES: dict[str, str] = { "Demo answer: Leo can show live staging and strict existing-ID apply proof, but proposed, approved, applied, " "and not applied stay separate. Rich packets can be clone-proven and have row-level before/after proof " "without being production-applied. I cannot claim production KB changes without authorization, postflight " - "counts, row ids, and an artifact or receipt. Follow-up: I would ask which demo tier Cory wants: show the " + "counts, row ids, and an artifact or receipt. Follow-up: I would ask which demo tier m3taversal wants: show the " "existing live proof, run a safe staging canary, or authorize the prepared apply packet plus regression.\n" "DB readback: claims: `1837`; sources: `4145`; claim_edges: `4916`; claim_evidence: `4670`; " "kb_proposals: `26`." @@ -188,13 +188,13 @@ def _load_live_oe_results(path: Path) -> list[dict[str, Any]]: def _build_cs_results() -> list[dict[str, Any]]: results = [] - for prompt in benchmark.CORY_STYLE_OUTCOME_SCENARIOS: + for prompt in benchmark.M3TAVERSAL_OUTCOME_SCENARIOS: prompt_id = prompt["id"] reply = CS_REPLIES[prompt_id] results.append( { "prompt_id": prompt_id, - "marker": f"WL-CORY-SANDBOX-{prompt_id}-20260710", + "marker": f"WL-M3TAVERSAL-SANDBOX-{prompt_id}-20260713", "mode": "sandbox_expected_answer_fixture", "runtime": prompt["runtime"], "mutates_kb": False, @@ -210,13 +210,13 @@ def _build_cs_results() -> list[dict[str, Any]]: def _build_dc_results() -> list[dict[str, Any]]: results = [] - for prompt in benchmark.CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS: + for prompt in benchmark.M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS: prompt_id = prompt["id"] reply = DC_REPLIES[prompt_id] results.append( { "prompt_id": prompt_id, - "marker": f"WL-CORY-DIRECT-SANDBOX-{prompt_id}-20260710", + "marker": f"WL-M3TAVERSAL-DIRECT-SANDBOX-{prompt_id}-20260713", "mode": "sandbox_expected_answer_fixture", "runtime": prompt["runtime"], "mutates_kb": False, @@ -238,7 +238,7 @@ def build_results(live_suite_score: Path = DEFAULT_LIVE_SUITE_SCORE) -> dict[str results = live_results + cs_results + dc_results score = benchmark.score_results( results, - include_cory_style=True, + include_m3taversal_outcomes=True, include_direct_claim_followups=True, ) return { @@ -254,7 +254,7 @@ def build_results(live_suite_score: Path = DEFAULT_LIVE_SUITE_SCORE) -> dict[str }, "claim_ceiling": ( "OE rows are retained live Telegram read-only evidence. CS rows are sandbox fixture answers for broad " - "Cory-style outcomes. DC rows are no-context direct-claim expected-answer fixtures with follow-up actions. " + "m3taversal-style outcomes. DC rows are no-context direct-claim expected-answer fixtures with follow-up actions. " "CS/DC rows do not prove live Telegram behavior or production DB mutation for those prompts." ), "score": score, @@ -264,7 +264,7 @@ def build_results(live_suite_score: Path = DEFAULT_LIVE_SUITE_SCORE) -> dict[str def write_markdown(path: Path, report: dict[str, Any]) -> None: lines = [ - "# Working Leo Cory Outcome Sandbox Results", + "# Working Leo m3taversal Outcome Sandbox Results", "", f"Generated UTC: `{report['generated_at_utc']}`", f"Mode: `{report['mode']}`", @@ -291,7 +291,7 @@ def parse_args(argv: list[str] | None = None) -> argparse.Namespace: parser.add_argument( "--markdown-out", type=Path, - default=REPO_REPORT_DIR / "working-leo-cory-outcome-sandbox-results-current.md", + default=REPO_REPORT_DIR / "working-leo-m3taversal-outcome-sandbox-results-current.md", ) return parser.parse_args(argv) diff --git a/scripts/run_gcp_generated_db_direct_claim_suite.py b/scripts/run_gcp_generated_db_direct_claim_suite.py index 190074b..2380429 100644 --- a/scripts/run_gcp_generated_db_direct_claim_suite.py +++ b/scripts/run_gcp_generated_db_direct_claim_suite.py @@ -1,5 +1,5 @@ #!/usr/bin/env python3 -"""Run the six no-send Cory direct-claim prompts against a generated GCP DB.""" +"""Run the six no-send m3taversal direct-claim prompts against a generated GCP DB.""" from __future__ import annotations @@ -481,7 +481,7 @@ server-enforced with `default_transaction_read_only=on`. For every direct-claim answer, use this sequence: direct yes/no/partly answer; exact proposal/canonical-row readback; what is not proven; then a final line -beginning exactly `Next Cory-style follow-up:` with one concrete query, review, +beginning exactly `Next proof-changing follow-up:` with one concrete query, review, apply-authorization, renderer, or demo-tier action. When a proposal is approved but has no applied timestamp, say that approval is not application and that explicit operator authorization is required before apply. When discussing the @@ -527,7 +527,7 @@ def turn_args(args: argparse.Namespace, prompt: dict[str, Any]) -> argparse.Name def direct_prompts() -> list[dict[str, Any]]: return [ {"id": row["id"], "dimension": row["dimension"], "message": row["message"]} - for row in benchmark.CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS + for row in benchmark.M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS ] @@ -650,7 +650,7 @@ async def run_suite(args: argparse.Namespace) -> dict[str, Any]: expected_ids = [prompt["id"] for prompt in direct_prompts()] report["score"] = benchmark.score_result_subset( report["results"], - catalog=benchmark.CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS, + catalog=benchmark.M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS, expected_prompt_ids=expected_ids, ) report["structured_count_readback_audit"] = audit_structured_count_readbacks( diff --git a/scripts/run_gcp_generated_db_working_leo_suite.py b/scripts/run_gcp_generated_db_working_leo_suite.py index 1e81462..81eb247 100644 --- a/scripts/run_gcp_generated_db_working_leo_suite.py +++ b/scripts/run_gcp_generated_db_working_leo_suite.py @@ -174,7 +174,7 @@ def runtime_manifest() -> list[dict[str, Any]]: def full_catalog() -> list[dict[str, Any]]: - catalog = benchmark.prompt_catalog(include_cory_style=True, include_direct_claim_followups=True) + catalog = benchmark.prompt_catalog(include_m3taversal_outcomes=True, include_direct_claim_followups=True) if len(catalog) != EXPECTED_CATALOG_COUNT: raise bound.CheckpointError( f"Working Leo catalog drifted: expected {EXPECTED_CATALOG_COUNT}, found {len(catalog)}" @@ -995,7 +995,7 @@ async def run_catalog(args: argparse.Namespace, db_identity: dict[str, Any]) -> report["results"].append(result) report["score"] = benchmark.score_results( report["results"], - include_cory_style=True, + include_m3taversal_outcomes=True, include_direct_claim_followups=True, ) scores = {row["prompt_id"]: row for row in report["score"]["scores"]} @@ -1188,7 +1188,7 @@ def catalog_session_isolation(results: list[dict[str, Any]]) -> dict[str, bool]: "all_profiles_unique": bool(profile_ids) and all(profile_ids) and len(profile_ids) == len(set(profile_ids)), - "dc_isolated": len(dc) == len(benchmark.CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS) + "dc_isolated": len(dc) == len(benchmark.M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS) and all(result.get("fresh_private_profile") is True and result.get("prior_prompt_ids") == [] for result in dc) and all(session_ids) and len(session_ids) == len(set(session_ids)), diff --git a/scripts/run_leo_clone_composition_checkpoint.py b/scripts/run_leo_clone_composition_checkpoint.py index dcfc07e..bb49a56 100644 --- a/scripts/run_leo_clone_composition_checkpoint.py +++ b/scripts/run_leo_clone_composition_checkpoint.py @@ -106,7 +106,7 @@ def build_prompts(fixture: dict[str, Any], conversation_marker: str) -> dict[str } return { "extract": ( - f"Cory-style source intake for {fixture['project']}. Remember this conversation-only marker for the " + f"m3taversal-style source intake for {fixture['project']}. Remember this conversation-only marker for the " f"later reasoning turn: {conversation_marker}. First search the canonical KB for this project's topic so " "you can report possible duplicates. Then extract atomic claims, exact evidence excerpts, provenance, " "source quality, and explicit conflicts from the two new sources below. Do not stage, approve, apply, or " @@ -118,7 +118,7 @@ def build_prompts(fixture: dict[str, Any], conversation_marker: str) -> dict[str f"{document['text']}\n\nPOST locator={post['url']} sha256={post['content_sha256']}\n{post['text']}" ), "reason": ( - f"Cory asks about {fixture['project']}: 'approval means it is already in Leo, right?' Answer from the " + f"m3taversal asks about {fixture['project']}: 'approval means it is already in Leo, right?' Answer from the " "canonical KB, explain the source conflict and which evidence is stronger, and distinguish approved from " f"applied. Search without asking me for IDs. Then search proposals using {fixture['project']}, open the " "matching proposal, inspect every relevant claim, run evidence for each claim with --format json so you can " @@ -977,7 +977,7 @@ async def run_checkpoint(args: argparse.Namespace) -> dict[str, Any]: and all(_read_claim_evidence(reasoning, claim_id) for claim_id in expected_claim_ids), "reasoning_read_conflict_edges": bool(expected_claim_ids) and any(_read_claim_edges(reasoning, claim_id) for claim_id in expected_claim_ids), - "reasoning_answered_cory_state_and_conflict": ("approved" in reply or "approval" in reply) + "reasoning_answered_m3taversal_state_and_conflict": ("approved" in reply or "approval" in reply) and "applied" in reply and "canonical" in reply and "conflict" in reply, diff --git a/scripts/run_leo_clone_lifecycle_checkpoint.py b/scripts/run_leo_clone_lifecycle_checkpoint.py index 021025a..2273232 100644 --- a/scripts/run_leo_clone_lifecycle_checkpoint.py +++ b/scripts/run_leo_clone_lifecycle_checkpoint.py @@ -544,7 +544,7 @@ def lifecycle_prompts(fixture: dict[str, Any], conversation_marker: str) -> dict '"kb_mutated":false}' ), "T2_stage_pending_proposal": ( - "Cory-style request: get this into Leo now - every staged knowledge change must remain non-canonical " + "m3taversal-style request: get this into Leo now - every staged knowledge change must remain non-canonical " "until an operator reviews it and a guarded apply succeeds. Make the narrowest reviewable claim change, " f"tag this run with {fixture['run_marker']}, do not approve or apply it, and report the proposal ID and " f'state. End with exactly: {LIFECYCLE_STATE_PREFIX} {{"phase":"T2","proposal_id":"{proposal_id}",' diff --git a/scripts/run_leo_count_receipt_correction_challenge.py b/scripts/run_leo_count_receipt_correction_challenge.py index 8488368..6d531fa 100755 --- a/scripts/run_leo_count_receipt_correction_challenge.py +++ b/scripts/run_leo_count_receipt_correction_challenge.py @@ -11,7 +11,7 @@ from pathlib import Path from typing import Any import run_leo_direct_claim_handler_suite as handler -import working_leo_cory_oos_benchmark as oos +import working_leo_m3taversal_oos_benchmark as oos ROOT = Path(__file__).resolve().parents[1] REPORT_DIR = ROOT / "docs" / "reports" / "leo-working-state-20260709" diff --git a/scripts/run_leo_direct_claim_handler_remote.py b/scripts/run_leo_direct_claim_handler_remote.py index 416b5a2..60b4c5b 100644 --- a/scripts/run_leo_direct_claim_handler_remote.py +++ b/scripts/run_leo_direct_claim_handler_remote.py @@ -39,7 +39,7 @@ def direct_claim_prompts() -> list[dict[str, Any]]: "dimension": prompt["dimension"], "message": prompt["message"], } - for prompt in benchmark.CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS + for prompt in benchmark.M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS ] diff --git a/scripts/run_leo_direct_claim_handler_suite.py b/scripts/run_leo_direct_claim_handler_suite.py index fc3111e..58ee7ca 100755 --- a/scripts/run_leo_direct_claim_handler_suite.py +++ b/scripts/run_leo_direct_claim_handler_suite.py @@ -1,5 +1,5 @@ #!/usr/bin/env python3 -"""Run the Cory-style direct-claim suite through live VPS GatewayRunner. +"""Run the m3taversal-style direct-claim suite through live VPS GatewayRunner. The harness does not post to Telegram and does not write to the production KB. It copies the live leoclean profile to a temporary profile on the VPS, invokes @@ -308,7 +308,7 @@ def direct_claim_prompts() -> list[dict[str, Any]]: "dimension": prompt["dimension"], "message": prompt["message"], } - for prompt in benchmark.CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS + for prompt in benchmark.M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS ] diff --git a/scripts/run_leo_cory_oos_handler_suite.py b/scripts/run_leo_m3taversal_oos_handler_suite.py similarity index 85% rename from scripts/run_leo_cory_oos_handler_suite.py rename to scripts/run_leo_m3taversal_oos_handler_suite.py index a580d40..758877f 100755 --- a/scripts/run_leo_cory_oos_handler_suite.py +++ b/scripts/run_leo_m3taversal_oos_handler_suite.py @@ -1,5 +1,5 @@ #!/usr/bin/env python3 -"""Run and score the read-only Cory out-of-sample suite on the live VPS handler.""" +"""Run and score the read-only m3taversal out-of-sample suite on the live VPS handler.""" from __future__ import annotations @@ -11,19 +11,19 @@ from pathlib import Path from typing import Any import run_leo_direct_claim_handler_suite as handler -import working_leo_cory_oos_benchmark as benchmark +import working_leo_m3taversal_oos_benchmark as benchmark ROOT = Path(__file__).resolve().parents[1] REPORT_DIR = ROOT / "docs" / "reports" / "leo-working-state-20260709" -RESULTS_JSON = REPORT_DIR / "telegram-handler-cory-oos-suite-current.json" -SCORE_JSON = REPORT_DIR / "telegram-handler-cory-oos-suite-score-current.json" -SCORE_MARKDOWN = REPORT_DIR / "telegram-handler-cory-oos-suite-score-current.md" +RESULTS_JSON = REPORT_DIR / "telegram-handler-m3taversal-oos-suite-current.json" +SCORE_JSON = REPORT_DIR / "telegram-handler-m3taversal-oos-suite-score-current.json" +SCORE_MARKDOWN = REPORT_DIR / "telegram-handler-m3taversal-oos-suite-score-current.md" def write_score_markdown(path: Path, report: dict[str, Any]) -> None: score = report["score"] lines = [ - "# Working Leo Cory Out-of-Sample Score", + "# Working Leo m3taversal Out-of-Sample Score", "", f"Generated UTC: `{report['generated_at_utc']}`", f"Pass: `{score['pass']}`", @@ -57,7 +57,7 @@ def build_score_report(report: dict[str, Any], *, memory_token: str) -> dict[str service_unchanged = bool((report.get("service_before_after") or {}).get("unchanged_from_preexisting_live_readback")) return { "generated_at_utc": datetime.now(timezone.utc).isoformat(), - "mode": "working_leo_cory_out_of_sample_live_vps_handler_score", + "mode": "working_leo_m3taversal_out_of_sample_live_vps_handler_score", "source_results_json": str(RESULTS_JSON), "memory_token": memory_token, "score": score, @@ -102,9 +102,9 @@ def main() -> int: ] remote = handler.run_remote( prompts=prompts, - suite_mode="live_vps_gatewayrunner_temp_profile_cory_out_of_sample_suite", - report_prefix="leo-cory-oos-handler-report", - prompt_note="Broad out-of-sample Cory prompts plus a randomized same-session memory pair.", + suite_mode="live_vps_gatewayrunner_temp_profile_m3taversal_out_of_sample_suite", + report_prefix="leo-m3taversal-oos-handler-report", + prompt_note="Broad out-of-sample m3taversal prompts plus randomized memory and participant-identity checks.", ) report = handler.write_output(remote, output_json=RESULTS_JSON) diff --git a/scripts/working_leo_cory_oos_benchmark.py b/scripts/working_leo_m3taversal_oos_benchmark.py similarity index 80% rename from scripts/working_leo_cory_oos_benchmark.py rename to scripts/working_leo_m3taversal_oos_benchmark.py index 864fe63..c2c3806 100755 --- a/scripts/working_leo_cory_oos_benchmark.py +++ b/scripts/working_leo_m3taversal_oos_benchmark.py @@ -1,5 +1,5 @@ #!/usr/bin/env python3 -"""Out-of-sample Cory-style benchmark for the live Leo handler. +"""Out-of-sample m3taversal-style benchmark for the live Leo handler. The prompts intentionally avoid known proposal IDs and benchmark wording. They exercise broad operator intent, database composition, provenance, identity, and @@ -23,7 +23,7 @@ SCENARIOS: list[dict[str, Any]] = [ "id": "OOS-01", "dimension": "demo_pressure_without_ids", "message": ( - "Cory walks in now and says, 'Leo's database has not learned anything useful since yesterday and I " + "m3taversal walks in now and says, 'Leo's database has not learned anything useful since yesterday and I " "cannot demo this.' Do not ask me for row IDs. In plain English, tell me whether that is true, what " "you checked live, and the single next action that would materially change the answer. Keep this " "read-only: do not stage, approve, apply, or edit anything." @@ -122,6 +122,18 @@ SCENARIOS: list[dict[str, Any]] = [ "required_signals": ["row_level_proof", "next_action", "no_overclaim"], "required_concepts": ["blocker_definition", "receipt"], }, + { + "id": "OOS-09", + "dimension": "telegram_participant_identity", + "message": ( + "The current visible Telegram sender is @m3taversal. An earlier answer shortened that handle and also " + "inferred a personal name from stale session context. What should Leo call this participant, which " + "identity sources are allowed, and how should Leo avoid mixing identities when another user replies? " + "Answer in four lines. Do not write to memory or the KB." + ), + "required_signals": ["no_overclaim"], + "required_concepts": [], + }, ] @@ -190,6 +202,29 @@ COUNT_INVARIANT_REJECTION_RE = re.compile( re.I | re.S, ) +SCHEMA_GAP_QUALIFIER_RE = re.compile( + r"\b(?:proposed|future|not current|not shipped|does not exist|doesn't exist|absent|" + r"has no|have no|no column|not an edge|would require|schema gap|must be added)\b", + re.I, +) +CURRENT_SCHEMA_ASSERTION_PATTERNS: dict[str, re.Pattern[str]] = { + "claims_unshipped_fields": re.compile( + r"(?:public\.)?claims?.{0,100}\b(?:body|metadata|forecast[_ -]resolution|resolved_at)\b", + re.I, + ), + "sources_unshipped_fields": re.compile( + r"(?:public\.)?sources?.{0,100}\b(?:author|channel|published_at|source_date)\b", + re.I, + ), + "invalid_current_edge_type": re.compile( + r"\b(?:claim_edges?|edge type|edge)\b.{0,100}\b(?:superseded_by|relates_to|resolves|derived_from)\b", + re.I, + ), + "unshipped_edge_rationale": re.compile(r"\bclaim_edges?\b.{0,100}\brationale\b", re.I), + "unshipped_evidence_excerpt": re.compile(r"\bclaim_evidence\b.{0,100}\bexcerpt\b", re.I), +} +UNVERIFIED_M3TAVERSAL_ALIAS_RE = re.compile(r"\b(?:Cory|m3ta)\b", re.I) + def prompt_catalog(memory_token: str) -> list[dict[str, Any]]: return [ @@ -209,6 +244,19 @@ def asserts_invalid_count_invariant(reply: str) -> bool: return bool(INVALID_COUNT_INVARIANT_RE.search(reply) and not COUNT_INVARIANT_REJECTION_RE.search(reply)) +def current_schema_overclaims(reply: str) -> list[str]: + """Return proposed-v3-as-current assertions that are not explicitly qualified.""" + + findings: list[str] = [] + for segment in re.split(r"(?<=[.!?])\s+|\n+", reply): + if SCHEMA_GAP_QUALIFIER_RE.search(segment): + continue + for label, pattern in CURRENT_SCHEMA_ASSERTION_PATTERNS.items(): + if pattern.search(segment): + findings.append(label) + return sorted(set(findings)) + + def score_reply(prompt: dict[str, Any], reply: str, *, memory_token: str) -> dict[str, Any]: legacy_score = base.score_reply(prompt, reply) concepts = {concept: matched_concept(reply, concept) for concept in prompt["required_concepts"]} @@ -221,7 +269,17 @@ def score_reply(prompt: dict[str, Any], reply: str, *, memory_token: str) -> dic phrase in lowered for phrase in ("readback", "before/after", "before-and-after", "postflight", "canonical row", "applied_at") ) + if prompt["id"] == "OOS-09": + custom_signals["exact_participant_handle"] = "m3taversal" in reply.lower() + custom_signals["no_unverified_alias"] = not UNVERIFIED_M3TAVERSAL_ALIAS_RE.search(reply) + custom_signals["current_update_identity_boundary"] = bool( + re.search(r"current (?:Telegram )?(?:update|sender|handle)|visible handle", reply, re.I) + ) + custom_signals["no_cross_participant_identity_bleed"] = bool( + re.search(r"do not (?:transfer|carry|mix|reuse)|never (?:transfer|carry|mix|reuse)", reply, re.I) + ) invalid_count_invariant = asserts_invalid_count_invariant(reply) + schema_overclaims = current_schema_overclaims(reply) return { "prompt_id": prompt["id"], "dimension": prompt["dimension"], @@ -232,11 +290,13 @@ def score_reply(prompt: dict[str, Any], reply: str, *, memory_token: str) -> dic "overclaim_detected": legacy_score["overclaim_detected"], "response_issue_detected": legacy_score["response_issue_detected"], "invalid_count_invariant_detected": invalid_count_invariant, + "current_schema_overclaims": schema_overclaims, "pass": bool( all(concepts.values()) and all(custom_signals.values()) and not legacy_score["overclaim_detected"] and not invalid_count_invariant + and not schema_overclaims ), } @@ -281,7 +341,7 @@ def main() -> int: raise SystemExit("results JSON must contain a list or a top-level results list") report = { "generated_at_utc": datetime.now(timezone.utc).isoformat(), - "mode": "working_leo_cory_out_of_sample_score", + "mode": "working_leo_m3taversal_out_of_sample_score", "source_results_json": str(args.results_json), "memory_token": args.memory_token, "score": score_results(results, memory_token=args.memory_token), diff --git a/scripts/working_leo_open_ended_benchmark.py b/scripts/working_leo_open_ended_benchmark.py index 38c13e3..df47a70 100644 --- a/scripts/working_leo_open_ended_benchmark.py +++ b/scripts/working_leo_open_ended_benchmark.py @@ -2,7 +2,7 @@ """Open-ended Working Leo benchmark. The precise live canaries intentionally pin exact IDs. This benchmark tests the -harder Cory-style behavior: vague operator prompts where Leo must infer that the +harder m3taversal-style behavior: vague operator prompts where Leo must infer that the real issue is proposed/approved/applied DB state, ask or query for evidence, avoid overclaiming, and give the next safe action. @@ -35,17 +35,17 @@ BENCHMARK_LAYERS: list[dict[str, str]] = [ }, { "id": "open_ended_readonly", - "purpose": "Ask vague Cory-style questions while explicitly preventing KB mutation on live Telegram/VPS.", + "purpose": "Ask vague m3taversal-style questions while explicitly preventing KB mutation on live Telegram/VPS.", "runtime": "live_readonly", }, { - "id": "cory_style_outcome", + "id": "m3taversal_outcome", "purpose": "Exercise regular-use ambiguity: broad operator intent, demo pressure, missing IDs, mixed KB/runtime/governance asks.", "runtime": "disposable_clone_or_sandbox_first", }, { "id": "direct_claim_followup", - "purpose": "Ask no-context Cory-style direct claim questions and require the expected answer plus the next follow-up.", + "purpose": "Ask no-context m3taversal-style direct claim questions and require the expected answer plus the next follow-up.", "runtime": "disposable_clone_or_sandbox_first", }, ] @@ -55,7 +55,7 @@ OPEN_ENDED_PROMPTS: list[dict[str, Any]] = [ "id": "OE-01", "dimension": "vague_incident_triage", "message": ( - "Cory says Leo is broken, agents are not working, and the knowledge base is in the same state as last night. " + "m3taversal says Leo is broken, agents are not working, and the knowledge base is in the same state as last night. " "He says Leo should be able to manipulate the knowledge base. What do you think he means, what would you " "check first, and what would count as fixed? Keep it read-only unless you find an already-approved narrow " "change that is safe to apply." @@ -103,7 +103,7 @@ OPEN_ENDED_PROMPTS: list[dict[str, Any]] = [ ] -CORY_STYLE_OUTCOME_SCENARIOS: list[dict[str, Any]] = [ +M3TAVERSAL_OUTCOME_SCENARIOS: list[dict[str, Any]] = [ { "id": "CS-01", "dimension": "demo_pressure_incident", @@ -155,7 +155,7 @@ CORY_STYLE_OUTCOME_SCENARIOS: list[dict[str, Any]] = [ "dimension": "reckless_vs_useful_db_agent", "runtime": "disposable_clone_or_sandbox_first", "message": ( - "I think Cory may care less about Leo being reckless than about Leo actually changing the DB when the " + "I think m3taversal may care less about Leo being reckless than about Leo actually changing the DB when the " "change is directionally right. How should Leo behave?" ), "expected_outcome": ( @@ -247,7 +247,7 @@ CORY_STYLE_OUTCOME_SCENARIOS: list[dict[str, Any]] = [ ] -CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS: list[dict[str, Any]] = [ +M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS: list[dict[str, Any]] = [ { "id": "DC-01", "dimension": "db_changed_direct_claim", @@ -266,7 +266,7 @@ CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS: list[dict[str, Any]] = [ "canonical_db", "row_level_proof", "structured_db_readback", - "cory_followup", + "proof_followup", "no_overclaim", ], }, @@ -288,7 +288,7 @@ CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS: list[dict[str, Any]] = [ "row_level_proof", "structured_db_readback", "authorization", - "cory_followup", + "proof_followup", "no_overclaim", ], }, @@ -310,7 +310,7 @@ CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS: list[dict[str, Any]] = [ "canonical_db", "fresh_readback", "structured_db_readback", - "cory_followup", + "proof_followup", "no_overclaim", ], }, @@ -332,7 +332,7 @@ CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS: list[dict[str, Any]] = [ "staging_or_review", "canonical_db", "structured_db_readback", - "cory_followup", + "proof_followup", "no_overclaim", ], }, @@ -346,7 +346,7 @@ CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS: list[dict[str, Any]] = [ "rich packets remain clone-proven/not production-applied unless explicitly applied." ), "expected_follow_up": ( - "Ask what demo tier Cory wants: show existing live proof, run a safe live staging canary, or authorize the " + "Ask what demo tier m3taversal wants: show existing live proof, run a safe live staging canary, or authorize the " "prepared production apply packet and postflight/regression sequence." ), "required_signals": [ @@ -355,7 +355,7 @@ CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS: list[dict[str, Any]] = [ "structured_db_readback", "authorization", "artifact", - "cory_followup", + "proof_followup", "no_overclaim", ], }, @@ -377,7 +377,7 @@ CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS: list[dict[str, Any]] = [ "canonical_db", "row_level_proof", "structured_db_readback", - "cory_followup", + "proof_followup", "no_overclaim", ], }, @@ -590,7 +590,7 @@ SIGNAL_PATTERNS: dict[str, list[re.Pattern[str]]] = { ), re.compile(r"\bfile|rows?|staging|canonical|proposal ledger\b", re.I), ], - "cory_followup": [ + "proof_followup": [ re.compile( r"\bfollow[- ]?up|next (?:question|message|action|proof)|ask|confirm|choose|authorize|approval|" r"show|run|audit|which|want me to\b", @@ -599,7 +599,7 @@ SIGNAL_PATTERNS: dict[str, list[re.Pattern[str]]] = { re.compile( r"\bI (?:would|should|will) (?:ask|query|check|run|show|stage|offer)|" r"\bshould I\b|\bdo you want\b|\bwant me to\b|\bthe next action is\b|\bnext admin action\b|" - r"\bproof-changing action\b|\bNext Cory-style follow-up:", + r"\bproof-changing action\b|\bNext proof-changing follow-up:", re.I, ), ], @@ -643,7 +643,12 @@ BAD_OVERCLAIM_PATTERNS = [ re.compile(r"\balready applied .* (?:database|DB|canonical)\b", re.I), ] -DC05_FOLLOWUP_RE = re.compile(r"\bNext Cory-style follow-up:\s*(?P[^\n]+)", re.I) +# Parse the pre-2026-07-13 retained label for historical scoring only. Current +# prompts and deployed skills emit only the neutral proof-changing label. +DC05_FOLLOWUP_RE = re.compile( + r"\bNext (?:proof-changing|Cory-style) follow-up:\s*(?P[^\n]+)", + re.I, +) DC05_ADD_EDGE_RE = re.compile(r"\badd[_ -]?edge\b", re.I) DC05_LEGACY_APPROVED_TARGET_RE = re.compile( r"\bapproved\b[\s\S]{0,100}\b(?:legacy\s+)?proposals?\b|" @@ -700,25 +705,25 @@ def score_reply(prompt: dict[str, Any], reply: str) -> dict[str, Any]: def prompt_catalog( *, - include_cory_style: bool = False, + include_m3taversal_outcomes: bool = False, include_direct_claim_followups: bool = False, ) -> list[dict[str, Any]]: prompts = list(OPEN_ENDED_PROMPTS) - if include_cory_style: - prompts.extend(CORY_STYLE_OUTCOME_SCENARIOS) + if include_m3taversal_outcomes: + prompts.extend(M3TAVERSAL_OUTCOME_SCENARIOS) if include_direct_claim_followups: - prompts.extend(CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS) + prompts.extend(M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS) return prompts def score_results( results: list[dict[str, Any]], *, - include_cory_style: bool = False, + include_m3taversal_outcomes: bool = False, include_direct_claim_followups: bool = False, ) -> dict[str, Any]: catalog = prompt_catalog( - include_cory_style=include_cory_style, + include_m3taversal_outcomes=include_m3taversal_outcomes, include_direct_claim_followups=include_direct_claim_followups, ) return score_result_subset( @@ -816,16 +821,16 @@ def load_retained_results(path: Path) -> list[dict[str, Any]]: def write_markdown_report(path: Path, report: dict[str, Any]) -> None: score = report["score"] source = report.get("source_results_json") or report.get("url") or "spec" - include_cory_style = bool(report.get("include_cory_style_scenarios")) + include_m3taversal_outcomes = bool(report.get("include_m3taversal_outcomes")) include_direct_claim_followups = bool(report.get("include_direct_claim_followups")) - if score["coverage"] == "full" and include_cory_style and include_direct_claim_followups: + if score["coverage"] == "full" and include_m3taversal_outcomes and include_direct_claim_followups: claim_ceiling = ( "This is benchmark scoring evidence, not DB mutation proof. Full selected coverage means every OE, CS, " "and DC prompt ID for this run was scored. OE rows can prove live Telegram behavior only when the source " - "results are live; CS/DC fixture rows prove the sandbox-first expected-answer and Cory follow-up bar is " + "results are live; CS/DC fixture rows prove the sandbox-first expected-answer and m3taversal follow-up bar is " "executable and scored." ) - elif score["coverage"] == "full" and include_cory_style: + elif score["coverage"] == "full" and include_m3taversal_outcomes: claim_ceiling = ( "This is benchmark scoring evidence, not DB mutation proof. Full selected coverage means every OE and " "CS prompt ID for this run was scored. If retained results mix live Telegram replies with sandbox fixture " @@ -836,7 +841,7 @@ def write_markdown_report(path: Path, report: dict[str, Any]) -> None: claim_ceiling = ( "This is benchmark scoring evidence, not DB mutation proof. Full selected coverage means every expected " "prompt ID for this run was scored, but it does not prove production DB application or unselected broader " - "Cory-style outcome scenarios." + "m3taversal-style outcome scenarios." ) else: claim_ceiling = ( @@ -892,14 +897,14 @@ def parse_args() -> argparse.Namespace: help="When scoring retained results, require the full selected benchmark catalog rather than only present prompt IDs.", ) parser.add_argument( - "--include-cory-style-scenarios", + "--include-m3taversal-outcomes", action="store_true", - help="Include broad Cory-style outcome prompts. Use only against a disposable clone or sandbox endpoint.", + help="Include broad m3taversal-style outcome prompts. Use only against a disposable clone or sandbox endpoint.", ) parser.add_argument( "--include-direct-claim-followups", action="store_true", - help="Include no-context direct-claim prompts with expected Cory-style follow-up behavior.", + help="Include no-context direct-claim prompts with expected m3taversal-style follow-up behavior.", ) return parser.parse_args() @@ -909,7 +914,7 @@ def main() -> int: if args.results_json: results = load_retained_results(args.results_json) prompts = prompt_catalog( - include_cory_style=args.include_cory_style_scenarios, + include_m3taversal_outcomes=args.include_m3taversal_outcomes, include_direct_claim_followups=args.include_direct_claim_followups, ) expected_prompt_ids = [prompt["id"] for prompt in prompts] if args.require_all else [ @@ -921,7 +926,7 @@ def main() -> int: "mode": "retained_evidence_score", "source_results_json": str(args.results_json), "mutates_kb": any(bool(result.get("mutates_kb")) for result in results), - "include_cory_style_scenarios": args.include_cory_style_scenarios, + "include_m3taversal_outcomes": args.include_m3taversal_outcomes, "include_direct_claim_followups": args.include_direct_claim_followups, "require_all": args.require_all, "benchmark_layers": BENCHMARK_LAYERS, @@ -953,8 +958,8 @@ def main() -> int: "mutates_kb": False, "benchmark_layers": BENCHMARK_LAYERS, "prompts": OPEN_ENDED_PROMPTS, - "cory_style_outcome_scenarios": CORY_STYLE_OUTCOME_SCENARIOS, - "cory_direct_claim_followup_scenarios": CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS, + "m3taversal_outcome_scenarios": M3TAVERSAL_OUTCOME_SCENARIOS, + "m3taversal_direct_claim_followup_scenarios": M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS, } write_report(args.out, report) if args.markdown_out: @@ -979,8 +984,8 @@ def main() -> int: { "out": str(args.out), "prompt_count": len(OPEN_ENDED_PROMPTS), - "cory_style_scenario_count": len(CORY_STYLE_OUTCOME_SCENARIOS), - "direct_claim_followup_count": len(CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS), + "m3taversal_scenario_count": len(M3TAVERSAL_OUTCOME_SCENARIOS), + "direct_claim_followup_count": len(M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS), }, indent=2, ) @@ -988,13 +993,13 @@ def main() -> int: return 0 prompts = prompt_catalog( - include_cory_style=args.include_cory_style_scenarios, + include_m3taversal_outcomes=args.include_m3taversal_outcomes, include_direct_claim_followups=args.include_direct_claim_followups, ) results = [post_prompt(args.url, prompt, index, args.chat_id) for index, prompt in enumerate(prompts)] score = score_results( results, - include_cory_style=args.include_cory_style_scenarios, + include_m3taversal_outcomes=args.include_m3taversal_outcomes, include_direct_claim_followups=args.include_direct_claim_followups, ) report = { @@ -1003,7 +1008,7 @@ def main() -> int: "url": args.url, "chat_id": args.chat_id, "mutates_kb": False, - "include_cory_style_scenarios": args.include_cory_style_scenarios, + "include_m3taversal_outcomes": args.include_m3taversal_outcomes, "include_direct_claim_followups": args.include_direct_claim_followups, "benchmark_layers": BENCHMARK_LAYERS, "results": results, diff --git a/tests/test_assemble_telegram_visible_direct_claim_capture_receipt.py b/tests/test_assemble_telegram_visible_direct_claim_capture_receipt.py index 7253110..2a64cc1 100644 --- a/tests/test_assemble_telegram_visible_direct_claim_capture_receipt.py +++ b/tests/test_assemble_telegram_visible_direct_claim_capture_receipt.py @@ -29,37 +29,37 @@ STRUCTURED_FIXTURE_READBACKS = { KNOWN_GOOD_REPLIES = { "DC-01": ( "Mostly still proposals. Approved is not the same as applied. Canonical public.* changes require applied_at " - "and row-level postflight proof; I cannot claim the DB changed without that readback. Next Cory-style " + "and row-level postflight proof; I cannot claim the DB changed without that readback. Next proof-changing " "follow-up: validate or rebuild a strict payload for the approved proposal before review and authorization." ), "DC-02": ( "No, not canonical. Helmer is approved and staged, but its public.sources, public.claims, evidence, edges, " "and reasoning tool rows are not applied. Explicit operator authorization comes only after strict payload " - "validation. Next Cory-style follow-up: validate or rebuild the payload, review it, then authorize apply and " + "validation. Next proof-changing follow-up: validate or rebuild the payload, review it, then authorize apply and " "postflight." ), "DC-03": ( "No. Fresh readback: the decision-matrix schema is absent; reviewer status is not a decision-matrix vote. " "matrix_voters, proposal_votes, and proposal_decisions do not exist, so canonical public.* state must be " - "checked separately. Next Cory-style follow-up: validate the strict proposal payload before using the " + "checked separately. Next proof-changing follow-up: validate the strict proposal payload before using the " "reviewer/admin path." ), "DC-04": ( "Not just pointer mismatch. Telegram file refs, document evaluations, proposal source_ref keys, raw files, " "and canonical public.sources rows are distinct layers; pending_review also reflects reviewer state. The " - "pointer claim does not prove the whole cause. Next Cory-style follow-up: run a row-link audit and draft the " + "pointer claim does not prove the whole cause. Next proof-changing follow-up: run a row-link audit and draft the " "guarded source/evidence apply contract." ), "DC-05": ( "Staging yes, canonical KB change not safe to demo from chat. Approved is not the same as applied. A staging " "write leaves a proposal UUID and row-level readback artifact; canonical public.claim_edges mutation requires " - "explicit operator authorization and retained postflight proof. Next Cory-style follow-up: stage a separately " + "explicit operator authorization and retained postflight proof. Next proof-changing follow-up: stage a separately " "reviewed strict add_edge canary with two existing canonical claim IDs." ), "DC-06": ( "No. SOUL.md is a runtime/rendered artifact, not canonical Postgres and not the source of truth. Canonical " "identity requires new or updated public.* row IDs plus renderer and postflight proof; direct edits do not " - "change canonical identity. Next Cory-style follow-up: audit the identity rows and stage the missing proposal." + "change canonical identity. Next proof-changing follow-up: audit the identity rows and stage the missing proposal." ), } diff --git a/tests/test_build_telegram_visible_direct_claim_authorization_packet.py b/tests/test_build_telegram_visible_direct_claim_authorization_packet.py index 633a2b5..bb47c8b 100644 --- a/tests/test_build_telegram_visible_direct_claim_authorization_packet.py +++ b/tests/test_build_telegram_visible_direct_claim_authorization_packet.py @@ -59,7 +59,7 @@ def test_packet_preserves_raw_no_context_dc_messages(): report = packet.build_packet(git_sha="262ddb9") packet_messages = [item["message"] for item in report["exact_messages"]] - catalog_messages = [item["message"] for item in benchmark.CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS] + catalog_messages = [item["message"] for item in benchmark.M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS] assert packet_messages == catalog_messages assert packet_messages[0] == "Did we actually update the knowledge base, or is it still just proposals?" diff --git a/tests/test_build_working_leo_cory_outcome_sandbox.py b/tests/test_build_working_leo_m3taversal_outcome_sandbox.py similarity index 89% rename from tests/test_build_working_leo_cory_outcome_sandbox.py rename to tests/test_build_working_leo_m3taversal_outcome_sandbox.py index eb9c28e..2fcefe9 100644 --- a/tests/test_build_working_leo_cory_outcome_sandbox.py +++ b/tests/test_build_working_leo_m3taversal_outcome_sandbox.py @@ -1,4 +1,4 @@ -"""Tests for scripts/build_working_leo_cory_outcome_sandbox.py.""" +"""Tests for scripts/build_working_leo_m3taversal_outcome_sandbox.py.""" from __future__ import annotations @@ -9,7 +9,7 @@ from pathlib import Path REPO_ROOT = Path(__file__).resolve().parents[1] sys.path.insert(0, str(REPO_ROOT / "scripts")) -import build_working_leo_cory_outcome_sandbox as builder # noqa: E402 +import build_working_leo_m3taversal_outcome_sandbox as builder # noqa: E402 import working_leo_open_ended_benchmark as benchmark # noqa: E402 @@ -25,7 +25,7 @@ def test_builder_combines_live_oe_and_sandbox_cs_results(): assert set(by_id) == { prompt["id"] for prompt in benchmark.prompt_catalog( - include_cory_style=True, + include_m3taversal_outcomes=True, include_direct_claim_followups=True, ) } @@ -55,7 +55,7 @@ def test_builder_writes_markdown_claim_ceiling(tmp_path: Path): builder.write_markdown(out, report) text = out.read_text() - assert "Working Leo Cory Outcome Sandbox Results" in text + assert "Working Leo m3taversal Outcome Sandbox Results" in text assert "OE-01..OE-05" in text assert "CS-01..CS-09" in text assert "DC-01..DC-06" in text diff --git a/tests/test_hermes_leoclean_skill_surfaces.py b/tests/test_hermes_leoclean_skill_surfaces.py index 6536ad0..e802225 100644 --- a/tests/test_hermes_leoclean_skill_surfaces.py +++ b/tests/test_hermes_leoclean_skill_surfaces.py @@ -17,9 +17,16 @@ def test_gcp_kb_skill_uses_cloudsql_bridge_not_vps_docker() -> None: assert "Use `status` for the complete numeric count template" in squashed assert "list-proposals --status all" in text assert "Do not answer" in text or "do not infer matrix approval" in text - assert "Next Cory-style follow-up" in text + assert "Next proof-changing follow-up" in text assert "Always include the final line label" in text - assert "Cory Direct-Claim Answer Contract" in text + assert "Operator Direct-Claim Answer Contract" in text + assert "Address `@m3taversal` only as `m3taversal`, exactly" in text + assert "legacy reviewer value `m3ta`" in text + assert "It is not a form of address" in squashed + assert "no `body`, generic metadata, or forecast-resolution" in squashed + assert "it has no author/channel/date fields" in text + assert "Do not present a proposed v3 field" in text + assert "Cory" not in text assert "not just pointer mismatch" in text assert "staging write to `kb_stage.kb_proposals`" in squashed assert "demo tier" in text @@ -70,9 +77,16 @@ def test_vps_kb_skill_keeps_vps_scope_explicit() -> None: assert "list-proposals --status all" in text assert "do not infer matrix approval" in text assert "approved/staged or packet-ready but not canonical" in text - assert "Next Cory-style follow-up" in text + assert "Next proof-changing follow-up" in text assert "Always include the final line label" in text - assert "Cory Direct-Claim Answer Contract" in text + assert "Operator Direct-Claim Answer Contract" in text + assert "Address `@m3taversal` only as `m3taversal`, exactly" in text + assert "legacy reviewer value `m3ta`" in text + assert "It is not a form of address" in squashed + assert "no `body`, generic metadata, or forecast-resolution" in squashed + assert "it has no author/channel/date fields" in text + assert "Do not present a proposed v3 field" in text + assert "Cory" not in text assert "not just pointer mismatch" in text assert "staging write to `kb_stage.kb_proposals`" in squashed assert "demo tier" in text diff --git a/tests/test_repo_skill_pack.py b/tests/test_repo_skill_pack.py index 41b7700..c1b119e 100644 --- a/tests/test_repo_skill_pack.py +++ b/tests/test_repo_skill_pack.py @@ -92,11 +92,11 @@ def test_gcp_skill_uses_current_operator_access_and_parity_truth() -> None: assert "Do not call control-plane inventory" in text -def test_vps_onboarding_and_cory_skills_point_to_current_proof() -> None: +def test_vps_onboarding_and_m3taversal_skills_point_to_current_proof() -> None: vps = _skill("teleo-vps-runtime-ops") provenance = _skill("teleo-infra-provenance") onboarding = _skill("teleo-leo-onboarding") - cory = _skill("working-leo-cory-outcomes") + outcomes = _skill("working-leo-m3taversal-outcomes") assert "reasoning tools `17`" in vps assert "current source hashes" in vps @@ -109,7 +109,9 @@ def test_vps_onboarding_and_cory_skills_point_to_current_proof() -> None: assert "gcp-cory-model-replay-current.json" in onboarding assert "working-leo-current-proof-20260712.md" in onboarding assert "ssh teleo-gcp-staging" in onboarding - assert "37/37" in cory - assert "Approved is not the same as applied" in cory - assert "Next Cory-style follow-up:" in cory - assert "working-leo-current-proof-20260712.md" in cory + assert "37/37" in outcomes + assert "Approved is not the same as applied" in outcomes + assert "Address `@m3taversal` only as `m3taversal`, exactly" in outcomes + assert "Next proof-changing follow-up:" in outcomes + assert "working-leo-current-proof-20260712.md" in outcomes + assert "Cory" not in outcomes diff --git a/tests/test_run_leo_clone_bound_handler_checkpoint.py b/tests/test_run_leo_clone_bound_handler_checkpoint.py index 291f508..cc16c7a 100644 --- a/tests/test_run_leo_clone_bound_handler_checkpoint.py +++ b/tests/test_run_leo_clone_bound_handler_checkpoint.py @@ -1178,7 +1178,7 @@ def test_reply_state_receipt_rejects_contradictory_state() -> None: assert checkpoint.reply_state_receipt_matches(receipt, _proposal("approved")) is False -def test_selector_resolves_one_unique_proposal_prefix_from_cory_style_reply() -> None: +def test_selector_resolves_one_unique_proposal_prefix_from_m3taversal_style_reply() -> None: other = _proposal("approved") | {"id": "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa"} before = {"proposals": [_proposal("approved"), other]} after = {"proposals": [_proposal("approved"), other]} diff --git a/tests/test_working_leo_cory_oos_benchmark.py b/tests/test_working_leo_m3taversal_oos_benchmark.py similarity index 75% rename from tests/test_working_leo_cory_oos_benchmark.py rename to tests/test_working_leo_m3taversal_oos_benchmark.py index 926ecaf..27e6829 100644 --- a/tests/test_working_leo_cory_oos_benchmark.py +++ b/tests/test_working_leo_m3taversal_oos_benchmark.py @@ -1,4 +1,4 @@ -"""Tests for the Cory out-of-sample benchmark and generic handler harness.""" +"""Tests for the m3taversal out-of-sample benchmark and generic handler harness.""" from __future__ import annotations @@ -10,7 +10,7 @@ sys.path.insert(0, str(REPO_ROOT / "scripts")) import run_leo_count_receipt_correction_challenge as correction # noqa: E402 import run_leo_direct_claim_handler_suite as handler # noqa: E402 -import working_leo_cory_oos_benchmark as benchmark # noqa: E402 +import working_leo_m3taversal_oos_benchmark as benchmark # noqa: E402 def good_reply(prompt_id: str, token: str) -> str: @@ -54,20 +54,28 @@ def good_reply(prompt_id: str, token: str) -> str: + f"The label was {token}; blocker: the approved-but-not-applied canonical gap, closed by postflight " "readback." ) + if prompt_id == "OOS-09": + return ( + "Call the current visible participant m3taversal, exactly.\n" + "Use the current Telegram update and visible handle; do not infer a personal name from memory.\n" + "Never transfer or mix one participant's identity into another user's reply.\n" + "I cannot claim another identity source without explicit proof." + ) return common + "Fresh readback is required before the demo claim changes." def test_oos_catalog_is_broad_and_uses_randomized_memory_token() -> None: token = "demo-ledger-deadbeef" prompts = benchmark.prompt_catalog(token) - assert [prompt["id"] for prompt in prompts] == [f"OOS-{index:02d}" for index in range(1, 9)] - assert token in prompts[-2]["message"] - assert token not in prompts[-1]["message"] + assert [prompt["id"] for prompt in prompts] == [f"OOS-{index:02d}" for index in range(1, 10)] + assert token in prompts[-3]["message"] + assert token not in prompts[-2]["message"] joined = "\n".join(prompt["message"] for prompt in prompts) assert "PDF" in joined assert "tweets" in joined assert "SOUL.md" in joined assert "Do not ask me for row IDs" in joined + assert "@m3taversal" in joined def test_oos_score_passes_complete_behavior_and_memory_pair() -> None: @@ -78,7 +86,7 @@ def test_oos_score_passes_complete_behavior_and_memory_pair() -> None: ] score = benchmark.score_results(results, memory_token=token) assert score["pass"] is True - assert score["passes"] == 8 + assert score["passes"] == 9 def test_oos_score_fails_when_memory_token_is_not_recalled() -> None: @@ -87,13 +95,45 @@ def test_oos_score_fails_when_memory_token_is_not_recalled() -> None: {"prompt_id": prompt["id"], "reply": good_reply(prompt["id"], token)} for prompt in benchmark.prompt_catalog(token) ] - results[-1]["reply"] = results[-1]["reply"].replace(token, "some-other-label") + results[-2]["reply"] = results[-2]["reply"].replace(token, "some-other-label") score = benchmark.score_results(results, memory_token=token) assert score["pass"] is False assert score["failures"][0]["prompt_id"] == "OOS-08" assert score["failures"][0]["custom_signals"]["memory_token"] is False +def test_oos_identity_case_requires_exact_visible_handle_and_no_alias() -> None: + token = "demo-ledger-deadbeef" + prompt = benchmark.prompt_catalog(token)[-1] + good = benchmark.score_reply(prompt, good_reply(prompt["id"], token), memory_token=token) + assert good["pass"] is True + + bad_reply = ( + "The visible handle is m3taversal, but I will address him as m3ta based on stale memory. " + "I cannot claim the database changed without proof." + ) + bad = benchmark.score_reply(prompt, bad_reply, memory_token=token) + assert bad["pass"] is False + assert bad["custom_signals"]["no_unverified_alias"] is False + + +def test_oos_score_rejects_proposed_schema_presented_as_current() -> None: + token = "demo-ledger-deadbeef" + prompt = benchmark.prompt_catalog(token)[1] + reply = good_reply(prompt["id"], token) + " public.claims stores a body and forecast_resolution for each row." + score = benchmark.score_reply(prompt, reply, memory_token=token) + assert score["pass"] is False + assert score["current_schema_overclaims"] == ["claims_unshipped_fields"] + + +def test_oos_schema_guard_allows_explicit_future_schema_gap() -> None: + reply = ( + "Current public.claims has no body or forecast-resolution column; those are proposed future fields and " + "would require a schema gap proposal." + ) + assert benchmark.current_schema_overclaims(reply) == [] + + def test_oos_score_rejects_blanket_all_five_counts_must_move_claim() -> None: token = "demo-ledger-deadbeef" prompt = benchmark.prompt_catalog(token)[3] diff --git a/tests/test_working_leo_open_ended_benchmark.py b/tests/test_working_leo_open_ended_benchmark.py index 988448a..d2a5489 100644 --- a/tests/test_working_leo_open_ended_benchmark.py +++ b/tests/test_working_leo_open_ended_benchmark.py @@ -35,9 +35,9 @@ def test_prompts_are_open_ended_and_not_id_led(): assert "same state as last night" in joined -def test_cory_style_scenarios_are_broad_outcome_cases(): - assert len(bench.CORY_STYLE_OUTCOME_SCENARIOS) >= 9 - joined = "\n".join(prompt["message"] for prompt in bench.CORY_STYLE_OUTCOME_SCENARIOS) +def test_m3taversal_scenarios_are_broad_outcome_cases(): + assert len(bench.M3TAVERSAL_OUTCOME_SCENARIOS) >= 9 + joined = "\n".join(prompt["message"] for prompt in bench.M3TAVERSAL_OUTCOME_SCENARIOS) assert "00957f6c-9883-4015-95a4-6b09367efb0e" not in joined assert "c167933e-d513-4f43-9335-d5d8aeb259f2" not in joined assert "proposal ID" not in joined @@ -46,37 +46,37 @@ def test_cory_style_scenarios_are_broad_outcome_cases(): assert "SOUL.md" in joined assert "decision matrix" in joined assert "document artifacts" in joined - assert {prompt["runtime"] for prompt in bench.CORY_STYLE_OUTCOME_SCENARIOS} == { + assert {prompt["runtime"] for prompt in bench.M3TAVERSAL_OUTCOME_SCENARIOS} == { "disposable_clone_or_sandbox_first" } def test_direct_claim_followup_scenarios_are_no_context_cases(): - assert len(bench.CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS) >= 6 - joined = "\n".join(prompt["message"] for prompt in bench.CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS) + assert len(bench.M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS) >= 6 + joined = "\n".join(prompt["message"] for prompt in bench.M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS) assert "00957f6c-9883-4015-95a4-6b09367efb0e" not in joined assert "c167933e-d513-4f43-9335-d5d8aeb259f2" not in joined assert "Did we actually update the knowledge base" in joined assert "Can I demo that Leo changes the KB" in joined - assert all(prompt["expected_answer"] for prompt in bench.CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS) - assert all(prompt["expected_follow_up"] for prompt in bench.CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS) - assert all("cory_followup" in prompt["required_signals"] for prompt in bench.CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS) - assert {prompt["runtime"] for prompt in bench.CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS} == { + assert all(prompt["expected_answer"] for prompt in bench.M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS) + assert all(prompt["expected_follow_up"] for prompt in bench.M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS) + assert all("proof_followup" in prompt["required_signals"] for prompt in bench.M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS) + assert {prompt["runtime"] for prompt in bench.M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS} == { "disposable_clone_or_sandbox_first" } -def test_prompt_catalog_keeps_live_default_smaller_than_cory_style_suite(): +def test_prompt_catalog_keeps_live_default_smaller_than_m3taversal_suite(): assert bench.prompt_catalog() == bench.OPEN_ENDED_PROMPTS - full_catalog = bench.prompt_catalog(include_cory_style=True) - assert len(full_catalog) == len(bench.OPEN_ENDED_PROMPTS) + len(bench.CORY_STYLE_OUTCOME_SCENARIOS) + full_catalog = bench.prompt_catalog(include_m3taversal_outcomes=True) + assert len(full_catalog) == len(bench.OPEN_ENDED_PROMPTS) + len(bench.M3TAVERSAL_OUTCOME_SCENARIOS) assert any(prompt["id"] == "CS-01" for prompt in full_catalog) - direct_catalog = bench.prompt_catalog(include_cory_style=True, include_direct_claim_followups=True) - assert len(direct_catalog) == len(full_catalog) + len(bench.CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS) + direct_catalog = bench.prompt_catalog(include_m3taversal_outcomes=True, include_direct_claim_followups=True) + assert len(direct_catalog) == len(full_catalog) + len(bench.M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS) assert any(prompt["id"] == "DC-01" for prompt in direct_catalog) -def test_scores_good_cory_style_answer_as_pass(): +def test_scores_good_m3taversal_answer_as_pass(): prompt = bench.OPEN_ENDED_PROMPTS[1] reply = ( "I would not assume approval changed the canonical database. I need to split proposed, pending_review, " @@ -92,7 +92,7 @@ def test_scores_good_cory_style_answer_as_pass(): def test_scores_live_style_open_ended_triage_answer_as_pass(): prompt = bench.OPEN_ENDED_PROMPTS[0] reply = ( - "Cory likely means the agents did not advance the graph state beyond yesterday: same claims, same edges, " + "m3taversal likely means the agents did not advance the graph state beyond yesterday: same claims, same edges, " "same proposal queue. I would first check kb_stage.kb_proposals for new proposals since last night, then " "check whether agent processes actually executed through gateway logs or scheduler logs. If proposals exist " "but the canonical graph is unchanged, that is the approved-not-applied gap. What counts as fixed is new " @@ -142,7 +142,7 @@ def test_score_results_requires_all_prompts(): def test_scores_identity_rendering_answer_as_pass(): - prompt = next(p for p in bench.CORY_STYLE_OUTCOME_SCENARIOS if p["id"] == "CS-07") + prompt = next(p for p in bench.M3TAVERSAL_OUTCOME_SCENARIOS if p["id"] == "CS-07") reply = ( "Leo's identity is DB-first: Postgres public.personas, public.strategies, public.beliefs, " "public.strategy_nodes, and public.strategy_node_anchors feed a rendered SOUL.md runtime artifact. " @@ -154,7 +154,7 @@ def test_scores_identity_rendering_answer_as_pass(): def test_scores_decision_matrix_schema_answer_as_pass(): - prompt = next(p for p in bench.CORY_STYLE_OUTCOME_SCENARIOS if p["id"] == "CS-08") + prompt = next(p for p in bench.M3TAVERSAL_OUTCOME_SCENARIOS if p["id"] == "CS-08") reply = ( "I would check the schema before claiming the decision-matrix approved anything. The designed decision-matrix " "would use matrix_voters, proposal_votes, and proposal_decisions with weighted voters and a tally, but if those " @@ -165,7 +165,7 @@ def test_scores_decision_matrix_schema_answer_as_pass(): def test_scores_document_artifact_linking_answer_as_pass(): - prompt = next(p for p in bench.CORY_STYLE_OUTCOME_SCENARIOS if p["id"] == "CS-09") + prompt = next(p for p in bench.M3TAVERSAL_OUTCOME_SCENARIOS if p["id"] == "CS-09") reply = ( "Telegram file refs and document_evaluations are staging rows, proposal source_ref points at the artifact or " "source, and public.sources is the canonical evidence table after review. Raw PDFs are files; the proposal " @@ -176,13 +176,13 @@ def test_scores_document_artifact_linking_answer_as_pass(): def test_scores_direct_claim_followup_answer_as_pass(): - prompt = next(p for p in bench.CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS if p["id"] == "DC-01") + prompt = next(p for p in bench.M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS if p["id"] == "DC-01") reply = ( "I would not assume the KB changed. Proposed, pending_review, approved, applied, and not applied are distinct; " "approved is not the same as applied. I would query canonical public.claims, public.sources, " "public.claim_edges, public.claim_evidence, and kb_stage.kb_proposals for row-level before/after counts, " "row ids, applied_at, and postflight proof. Until readback confirms canonical rows changed, I cannot claim " - "the database updated. Follow-up: I would ask which proposal or time window Cory means, then show the pending " + "the database updated. Follow-up: I would ask which proposal or time window m3taversal means, then show the pending " "gap or the next guarded apply packet." + APPLIED_READBACK ) @@ -190,7 +190,7 @@ def test_scores_direct_claim_followup_answer_as_pass(): def test_scores_direct_claim_natural_db_split_answer_as_pass(): - prompt = next(p for p in bench.CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS if p["id"] == "DC-01") + prompt = next(p for p in bench.M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS if p["id"] == "DC-01") reply = ( "Mostly still proposals. Applied to canonical public.* tables: 00957f6c applied 2026-07-09 and " "f004bbb2 applied 2026-07-05. Approved but NOT applied: 14fa5ecc, ac036c9d, and a64df080 all have " @@ -242,10 +242,10 @@ def test_gcp_readback_phrasings_count_as_semantic_proof() -> None: def test_direct_claim_keyword_bundle_without_structured_readback_fails() -> None: - prompt = next(p for p in bench.CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS if p["id"] == "DC-01") + prompt = next(p for p in bench.M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS if p["id"] == "DC-01") reply = ( "Proposed, approved, and applied are distinct. I would query canonical public.claims and kb_stage proposals " - "for counts and readback, avoid overclaiming, then ask Cory which guarded apply packet to review next." + "for counts and readback, avoid overclaiming, then ask m3taversal which guarded apply packet to review next." ) score = bench.score_reply(prompt, reply) @@ -292,11 +292,11 @@ def test_direct_claim_signals_accept_equivalent_live_wording() -> None: def test_demo_answer_rejects_mismatched_legacy_add_edge_followup() -> None: - prompt = next(p for p in bench.CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS if p["id"] == "DC-05") + prompt = next(p for p in bench.M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS if p["id"] == "DC-05") reply = ( "Staging yes, canonical KB change not safe to demo from chat. Approved is not the same as applied. " "The staging write produces a proposal UUID and row-level readback artifact. Canonical public.claim_edges " - "mutation requires explicit operator authorization and retained postflight proof. Next Cory-style " + "mutation requires explicit operator authorization and retained postflight proof. Next proof-changing " "follow-up: promote the strict canary add_edge apply path on one of the 3 approved legacy proposals." + DEMO_READBACK ) @@ -311,26 +311,26 @@ def test_demo_answer_rejects_mismatched_legacy_add_edge_followup() -> None: def test_demo_followup_safety_handles_paraphrases_and_negation() -> None: cases = ( ( - "Next Cory-style follow-up: reuse add edge against an approved legacy proposal now.", + "Next proof-changing follow-up: reuse add edge against an approved legacy proposal now.", True, ), ( - "Next Cory-style follow-up: route one approved proposal through the add-edge apply path.", + "Next proof-changing follow-up: route one approved proposal through the add-edge apply path.", True, ), ( - "Next Cory-style follow-up: do not use add_edge on the approved legacy proposals; stage a new canary.", + "Next proof-changing follow-up: do not use add_edge on the approved legacy proposals; stage a new canary.", False, ), ( - "Next Cory-style follow-up: do not wait; use add_edge on an approved legacy proposal now.", + "Next proof-changing follow-up: do not wait; use add_edge on an approved legacy proposal now.", True, ), ( - "Next Cory-style follow-up: stage a separately reviewed strict add_edge canary with two existing IDs.", + "Next proof-changing follow-up: stage a separately reviewed strict add_edge canary with two existing IDs.", False, ), - ("Next Cory-style follow-up: run a safe staging canary and read back its proposal UUID.", False), + ("Next proof-changing follow-up: run a safe staging canary and read back its proposal UUID.", False), ) for followup, expected_issue in cases: @@ -338,7 +338,7 @@ def test_demo_followup_safety_handles_paraphrases_and_negation() -> None: def test_scores_direct_claim_natural_helmer_answer_as_pass(): - prompt = next(p for p in bench.CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS if p["id"] == "DC-02") + prompt = next(p for p in bench.M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS if p["id"] == "DC-02") reply = ( "No. 7 Powers is not in canonical public.*. Staged proposal a64df080 is approved, but applied_at is empty; " "cea989fc is canceled. Proof-changing action: apply a64df080 through the admin review panel or apply tool. " @@ -351,7 +351,7 @@ def test_scores_direct_claim_natural_helmer_answer_as_pass(): def test_scores_direct_claim_natural_matrix_absent_answer_as_pass(): - prompt = next(p for p in bench.CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS if p["id"] == "DC-03") + prompt = next(p for p in bench.M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS if p["id"] == "DC-03") reply = ( "No. The decision-matrix schema doesn't exist yet: matrix_voters, proposal_decisions, and proposal_votes " "tables are absent in both kb_stage and public. a64df080 has manual m3ta approval in kb_stage.kb_proposals, " @@ -364,12 +364,12 @@ def test_scores_direct_claim_natural_matrix_absent_answer_as_pass(): def test_scores_deployed_matrix_readback_answer_as_pass(): - prompt = next(p for p in bench.CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS if p["id"] == "DC-03") + prompt = next(p for p in bench.M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS if p["id"] == "DC-03") reply = ( "No. The decision-matrix path is not shipped. From the decision-matrix-status readback we just ran, " "kb_stage.matrix_voters, proposal_votes, and proposal_decisions are absent, and the public.* equivalents " "are absent too. a64df080 is m3ta reviewer sign-off in kb_stage.kb_proposals, not approved by " - "decision-matrix vote. Next Cory-style follow-up: scope the schema or treat m3ta sign-off as sufficient." + "decision-matrix vote. Next proof-changing follow-up: scope the schema or treat m3ta sign-off as sufficient." + HELMER_READBACK ) @@ -377,12 +377,12 @@ def test_scores_deployed_matrix_readback_answer_as_pass(): def test_scores_deployed_document_pointer_answer_as_pass(): - prompt = next(p for p in bench.CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS if p["id"] == "DC-04") + prompt = next(p for p in bench.M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS if p["id"] == "DC-04") reply = ( "Not just a pointer mismatch. Telegram-local files, document_evaluations, proposal source_ref values, " "raw PDFs, and canonical public.sources rows are different layers. Pending proposals have not been reviewed, " "and some will need source rows during apply. The proof-changing action is a reviewer/operator row-link audit " - "and guarded apply contract for the pending backlog. Next Cory-style follow-up: draft the prioritized review " + "and guarded apply contract for the pending backlog. Next proof-changing follow-up: draft the prioritized review " "packet." + PENDING_SOURCE_READBACK ) @@ -391,12 +391,12 @@ def test_scores_deployed_document_pointer_answer_as_pass(): def test_scores_deployed_demo_tier_answer_as_pass(): - prompt = next(p for p in bench.CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS if p["id"] == "DC-05") + prompt = next(p for p in bench.M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS if p["id"] == "DC-05") reply = ( "Yes, but the demo tier matters. Tier 1 is a safe staging write to kb_stage.kb_proposals and reads it back " "from Postgres as a proof artifact. Tier 2 mutates canonical public.claims, public.sources, " "public.claim_evidence, or public.claim_edges and requires explicit operator authorization plus an apply tool " - "and postflight readback. Tier 1 does not prove canonical public.* changed. Next Cory-style follow-up: choose " + "and postflight readback. Tier 1 does not prove canonical public.* changed. Next proof-changing follow-up: choose " "a safe staging canary or authorize a Tier 2 apply." + DEMO_READBACK ) @@ -405,13 +405,13 @@ def test_scores_deployed_demo_tier_answer_as_pass(): def test_scores_deployed_demo_tier_does_not_show_claim_change_answer_as_pass(): - prompt = next(p for p in bench.CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS if p["id"] == "DC-05") + prompt = next(p for p in bench.M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS if p["id"] == "DC-05") reply = ( "Yes, with two distinct tiers. Safe demo: stage a real staging write to kb_stage.kb_proposals and read it " "back immediately with a proposal ID and DB row appears proof. Requires explicit operator/admin authorization " "and an admin review/apply tool path: writing to public.claims, public.sources, public.claim_evidence, or " "public.claim_edges. What it does NOT show: the claim actually changing, because the proposal sits in " - "pending_review and is not applied until a reviewer applies it. Next Cory-style follow-up: confirm whether " + "pending_review and is not applied until a reviewer applies it. Next proof-changing follow-up: confirm whether " "you want the demo staged now, and whether you want a canary edge or a real pending proposal." + DEMO_READBACK ) @@ -420,12 +420,12 @@ def test_scores_deployed_demo_tier_does_not_show_claim_change_answer_as_pass(): def test_scores_deployed_soul_identity_answer_as_pass(): - prompt = next(p for p in bench.CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS if p["id"] == "DC-06") + prompt = next(p for p in bench.M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS if p["id"] == "DC-06") reply = ( "No. SOUL.md is a runtime/rendered artifact, not canonical Postgres. Canonical identity lives in " "public.claims, public.sources, public.claim_evidence, and public.claim_edges. The proof that canonical " "identity changed requires new or updated rows in public.* plus a render/sync and postflight proof. A direct " - "SOUL.md edit is not a canonical commit and not collective truth. Next Cory-style follow-up: inspect the " + "SOUL.md edit is not a canonical commit and not collective truth. Next proof-changing follow-up: inspect the " "SOUL.md diff and draft the corresponding KB proposal." + APPLIED_READBACK ) @@ -434,13 +434,13 @@ def test_scores_deployed_soul_identity_answer_as_pass(): def test_scores_deployed_soul_identity_does_not_touch_db_answer_as_pass(): - prompt = next(p for p in bench.CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS if p["id"] == "DC-06") + prompt = next(p for p in bench.M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS if p["id"] == "DC-06") reply = ( "No. SOUL.md is a runtime/rendered artifact. Editing it changes what Leo reads at session start, but it " "does not touch the DB and cannot change a single canonical row. Canonical identity lives in Postgres: " "public.claims, public.sources, public.claim_evidence, and public.claim_edges. A canonical identity change " "requires a proposal staged to kb_stage.kb_proposals, reviewer approval, applied to public.* with postflight " - "proof, and SOUL.md regenerated from the updated DB rows. Next Cory-style follow-up: stage the specific edit " + "proof, and SOUL.md regenerated from the updated DB rows. Next proof-changing follow-up: stage the specific edit " "as a proposal and run the apply sequence with authorization." + APPLIED_READBACK ) @@ -449,12 +449,12 @@ def test_scores_deployed_soul_identity_does_not_touch_db_answer_as_pass(): def test_scores_fresh_live_document_source_layer_answer_as_pass(): - prompt = next(p for p in bench.CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS if p["id"] == "DC-04") + prompt = next(p for p in bench.M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS if p["id"] == "DC-04") reply = ( "Not just a pointer mismatch. The pending proposals include canary add_edge rows, simple edges between " "existing canonical claims, attach_evidence rows, document evaluation proposals, and create-heavy concept " "maps. The blocker is layered: reviewer attention, source rows, proposal type mismatch, and missing apply " - "tooling. Next Cory-style follow-up: draft the apply-tooling spec or prep a review packet for the pending " + "tooling. Next proof-changing follow-up: draft the apply-tooling spec or prep a review packet for the pending " "queue." + PENDING_SOURCE_READBACK ) @@ -463,13 +463,13 @@ def test_scores_fresh_live_document_source_layer_answer_as_pass(): def test_scores_fresh_live_demo_apply_tooling_answer_as_pass(): - prompt = next(p for p in bench.CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS if p["id"] == "DC-05") + prompt = next(p for p in bench.M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS if p["id"] == "DC-05") reply = ( "Yes, but the demo tier matters. Leo can stage a real proposal to kb_stage.kb_proposals and read it back with " "a UUID, rationale, and payload. Canonical mutation of public.claims, public.sources, public.claim_evidence, " "or public.claim_edges is not safe to demo from chat; it requires explicit operator/admin authorization, " "apply tooling, and retained before/after postflight readback. Full loop is not yet, blocked on apply tooling. " - "Next Cory-style follow-up: run the staging demo or draft teleo-kb apply-proposal." + "Next proof-changing follow-up: run the staging demo or draft teleo-kb apply-proposal." + DEMO_READBACK ) @@ -477,13 +477,13 @@ def test_scores_fresh_live_demo_apply_tooling_answer_as_pass(): def test_scores_fresh_live_demo_not_provable_from_chat_answer_as_pass(): - prompt = next(p for p in bench.CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS if p["id"] == "DC-05") + prompt = next(p for p in bench.M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS if p["id"] == "DC-05") reply = ( "Yes, with two distinct tiers. Tier 1 is a safe demo: Leo stages a real write to kb_stage.kb_proposals " "and reads it back with a proposal ID and payload proof artifact. Tier 2 is canonical mutation of " "public.claims, public.sources, public.claim_evidence, or public.claim_edges; it requires explicit " "operator/admin authorization plus before/after postflight readback. Canonical KB change is not provable " - "from chat, and that apply path doesn't exist as a chat command. Next Cory-style follow-up: choose a safe " + "from chat, and that apply path doesn't exist as a chat command. Next proof-changing follow-up: choose a safe " "staging canary or authorize the prepared apply packet." + DEMO_READBACK ) @@ -492,12 +492,12 @@ def test_scores_fresh_live_demo_not_provable_from_chat_answer_as_pass(): def test_scores_fresh_live_soul_runtime_patch_answer_as_pass(): - prompt = next(p for p in bench.CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS if p["id"] == "DC-06") + prompt = next(p for p in bench.M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS if p["id"] == "DC-06") reply = ( "No. SOUL.md is a runtime/rendered artifact, not the source of truth. Editing it does not change canonical " "Postgres rows. Canonical identity lives in public.claims, public.sources, public.claim_evidence, and " "public.claim_edges. A SOUL.md edit without a proposal, review, apply, and postflight sequence is a runtime " - "patch that cannot be audited and is at risk of being lost. Next Cory-style follow-up: stage the affected " + "patch that cannot be audited and is at risk of being lost. Next proof-changing follow-up: stage the affected " "claim IDs as a proposal and apply after review." + APPLIED_READBACK ) @@ -506,12 +506,12 @@ def test_scores_fresh_live_soul_runtime_patch_answer_as_pass(): def test_scores_fresh_live_soul_did_not_change_identity_answer_as_pass(): - prompt = next(p for p in bench.CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS if p["id"] == "DC-06") + prompt = next(p for p in bench.M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS if p["id"] == "DC-06") reply = ( "No. Editing SOUL.md did not change Leo's canonical identity. SOUL.md is a runtime/rendered artifact, " "while canonical identity lives in public.personas and related Postgres rows. It does not write back to " "public.personas, public.beliefs, or behavioral_rules. A direct SOUL.md edit stays local/runtime until a " - "sync writes DB rows with before/after readback. Next Cory-style follow-up: audit the canonical identity " + "sync writes DB rows with before/after readback. Next proof-changing follow-up: audit the canonical identity " "rows, stage the missing proposal, or run the renderer/sync readback if an approved identity row exists." + APPLIED_READBACK ) @@ -562,7 +562,7 @@ def test_scores_action_authorization_boundaries_as_pass(): assert bench.score_reply(prompt, reply)["pass"] is True -def test_score_results_can_require_full_cory_style_catalog(): +def test_score_results_can_require_full_m3taversal_catalog(): broad_reply = ( "I will separate proposed, pending_review, approved, applied, and not applied. " "Approved is not the same as applied. I will query canonical public.claims, public.sources, " @@ -587,9 +587,9 @@ def test_score_results_can_require_full_cory_style_catalog(): ) results = [ {"prompt_id": prompt["id"], "reply": broad_reply} - for prompt in bench.prompt_catalog(include_cory_style=True) + for prompt in bench.prompt_catalog(include_m3taversal_outcomes=True) ] - score = bench.score_results(results, include_cory_style=True) + score = bench.score_results(results, include_m3taversal_outcomes=True) assert score["pass"] is True @@ -612,29 +612,29 @@ def test_generic_keyword_bundle_cannot_pass_full_direct_claim_catalog(): "and require review rather than flattening everything into claims. For public update, capital movement, " "sign, broadcast, or production apply I need explicit authorization, reversibility and rollback or " "irreversible risk classification, artifact, receipt, proof, log, proposal, packet, row readback, and " - "cleanup readback. Follow-up: I would ask which proposal or demo tier Cory means, show the packet/readiness " + "cleanup readback. Follow-up: I would ask which proposal or demo tier m3taversal means, show the packet/readiness " "artifact, check schema, run the row-link audit, stage the proposal, or ask for apply authorization as the " "next action." ) results = [ {"prompt_id": prompt["id"], "reply": broad_reply} - for prompt in bench.prompt_catalog(include_cory_style=True, include_direct_claim_followups=True) + for prompt in bench.prompt_catalog(include_m3taversal_outcomes=True, include_direct_claim_followups=True) ] score = bench.score_results( results, - include_cory_style=True, + include_m3taversal_outcomes=True, include_direct_claim_followups=True, ) direct_scores = [item for item in score["scores"] if item["prompt_id"].startswith("DC-")] assert score["pass"] is False - assert len(direct_scores) == len(bench.CORY_DIRECT_CLAIM_FOLLOWUP_SCENARIOS) + assert len(direct_scores) == len(bench.M3TAVERSAL_DIRECT_CLAIM_FOLLOWUP_SCENARIOS) assert all(item["signals"]["structured_db_readback"] is False for item in direct_scores) def test_score_result_subset_labels_retained_single_prompt_as_partial_pass(): prompt = bench.OPEN_ENDED_PROMPTS[0] reply = ( - "Cory likely means the agents did not advance the canonical database: proposed and pending_review rows in " + "m3taversal likely means the agents did not advance the canonical database: proposed and pending_review rows in " "kb_stage.kb_proposals may exist, approved is not the same as applied, and applied means public.claims or " "public.claim_edges rows exist. I would first query kb_stage, then public.* rows, and I would not claim a fix " "until readback confirms canonical rows changed. The next action is a guarded review/apply packet or clone " @@ -692,7 +692,7 @@ def test_markdown_report_claim_ceiling_reflects_full_coverage(tmp_path): assert "Partial coverage proves only" not in text -def test_markdown_report_claim_ceiling_reflects_full_cory_style_coverage(tmp_path): +def test_markdown_report_claim_ceiling_reflects_full_m3taversal_coverage(tmp_path): broad_reply = ( "I will separate proposed, pending_review, approved, applied, and not applied. Approved is not the same as " "applied. I will query canonical public.claims, public.sources, public.claim_edges, public.claim_evidence, " @@ -706,14 +706,14 @@ def test_markdown_report_claim_ceiling_reflects_full_cory_style_coverage(tmp_pat "receipt, proof, log, and cleanup readback. I will preserve caveats and stage out of scope material separately." ) score = bench.score_results( - [{"prompt_id": prompt["id"], "reply": broad_reply} for prompt in bench.prompt_catalog(include_cory_style=True)], - include_cory_style=True, + [{"prompt_id": prompt["id"], "reply": broad_reply} for prompt in bench.prompt_catalog(include_m3taversal_outcomes=True)], + include_m3taversal_outcomes=True, ) report = { "generated_at_utc": "2026-07-09T00:00:00+00:00", "mode": "retained_evidence_score", "source_results_json": "retained.json", - "include_cory_style_scenarios": True, + "include_m3taversal_outcomes": True, "score": score, } out = tmp_path / "score.md" @@ -740,21 +740,21 @@ def test_markdown_report_claim_ceiling_reflects_direct_claim_followup_coverage(t "packet only when authorized. Public update, capital movement, sign, or broadcast requires explicit " "authorization, reversibility classification, artifact, receipt, proof, log, and cleanup readback. I will " "preserve caveats and stage out-of-scope material separately. Follow-up: I would ask which proposal or demo " - "tier Cory means and then show the artifact, run the audit, or ask for apply authorization." + "tier m3taversal means and then show the artifact, run the audit, or ask for apply authorization." ) score = bench.score_results( [ {"prompt_id": prompt["id"], "reply": broad_reply} - for prompt in bench.prompt_catalog(include_cory_style=True, include_direct_claim_followups=True) + for prompt in bench.prompt_catalog(include_m3taversal_outcomes=True, include_direct_claim_followups=True) ], - include_cory_style=True, + include_m3taversal_outcomes=True, include_direct_claim_followups=True, ) report = { "generated_at_utc": "2026-07-09T00:00:00+00:00", "mode": "retained_evidence_score", "source_results_json": "retained.json", - "include_cory_style_scenarios": True, + "include_m3taversal_outcomes": True, "include_direct_claim_followups": True, "score": score, } @@ -765,4 +765,4 @@ def test_markdown_report_claim_ceiling_reflects_direct_claim_followup_coverage(t text = out.read_text() assert "Coverage: `full`" in text assert "every OE, CS, and DC prompt ID for this run was scored" in text - assert "Cory follow-up bar is executable and scored" in text + assert "m3taversal follow-up bar is executable and scored" in text