Generate ready-not-executed operator runbook for the integrated apply packet set.
Extend Cory-style benchmark with no-context direct-claim prompts and expected follow-ups.
Refresh retained reports and tests for the 20-prompt mixed-evidence score.
- Run OE-01 through OE-05 live Telegram read-only benchmark coverage
- Retain full-suite scoring artifacts and no-mutation safety readback
- Broaden scorer tests for live Leo wording without weakening claim ceilings
Files:
- Rescore retained live Telegram replies with partial versus full coverage labels
- Retain OE-01 benchmark score artifacts without mutating Leo or production DB
- Update Working Leo truth docs and tests so partial proof is not overclaimed
`docs/reports/leo-working-state-20260709/current-truth-index.md`
`docs/reports/leo-working-state-20260709/telegram-live-open-ended-benchmark-score-current.json`
`docs/reports/leo-working-state-20260709/telegram-live-open-ended-benchmark-score-current.md`
`docs/reports/leo-working-state-20260709/working-leo-current-state-20260709.md`
`docs/reports/leo-working-state-20260709/working-leo-execution-plan-current.md`
`scripts/working_leo_open_ended_benchmark.py`
`tests/test_working_leo_open_ended_benchmark.py`
- Create guarded Rio strategy nodes and anchors after the mapped base packet
- Prove clone apply and rollback while production rows and Leo runtime stay unchanged
- Retain packet docs, state image, and focused tests for future apply review
`docs/reports/leo-working-state-20260709/current-truth-index.md`
`docs/reports/leo-working-state-20260709/leo-db-state-23-rio-strategy-context.svg`
`docs/reports/leo-working-state-20260709/rio-strategy-apply-authorized-commit.sql`
`docs/reports/leo-working-state-20260709/rio-strategy-apply-rollback-rehearsal.sql`
`docs/reports/leo-working-state-20260709/rio-strategy-clone-rehearsal-current.log`
`docs/reports/leo-working-state-20260709/rio-strategy-context-current.md`
`docs/reports/leo-working-state-20260709/rio-strategy-delete-rollback.sql`
`docs/reports/leo-working-state-20260709/rio-strategy-packet.json`
`docs/reports/leo-working-state-20260709/rio-strategy-postflight.sql`
`docs/reports/leo-working-state-20260709/rio-strategy-preflight.sql`
`docs/reports/leo-working-state-20260709/working-leo-current-state-20260709.md`
`docs/reports/leo-working-state-20260709/working-leo-execution-plan-current.md`
`scripts/kb_rio_strategy_context_packet.py`
`tests/test_kb_rio_strategy_context_packet.py`
- Add broad Cory-style outcome scenarios to the open-ended benchmark so regular-use ambiguity is tested, not only exact ID canaries.
- Add a clone-proven cross-surface strategy anchor packet that updates the mapped rich proposal anchor only after Claim D exists.
- Retain VPS Docker DB, claim/body metadata, and cross-surface proof reports with ruff plus 37 focused tests passing.
`docs/reports/leo-working-state-20260709/cory-expected-working-leo-outcomes-20260709.md`
`docs/reports/leo-working-state-20260709/cross-surface-apply-authorized-commit.sql`
`docs/reports/leo-working-state-20260709/cross-surface-clone-rehearsal-current.log`
`docs/reports/leo-working-state-20260709/cross-surface-clone-rehearsal-failed-source-id-constraint.log`
`docs/reports/leo-working-state-20260709/cross-surface-delete-rollback.sql`
`docs/reports/leo-working-state-20260709/cross-surface-packet.json`
`docs/reports/leo-working-state-20260709/cross-surface-postflight.sql`
`docs/reports/leo-working-state-20260709/cross-surface-preflight.sql`
`docs/reports/leo-working-state-20260709/cross-surface-resolution-current.md`
`docs/reports/leo-working-state-20260709/current-truth-index.md`
`docs/reports/leo-working-state-20260709/leo-db-state-17-cory-outcome-benchmark.svg`
`docs/reports/leo-working-state-20260709/leo-db-state-18-db-vs-workspace.svg`
`docs/reports/leo-working-state-20260709/leo-db-state-19-claims-body-metadata.svg`
`docs/reports/leo-working-state-20260709/leo-db-state-20-cross-surface-anchor.svg`
`docs/reports/leo-working-state-20260709/working-leo-current-state-20260709.md`
`docs/reports/leo-working-state-20260709/working-leo-definition-20260709.md`
`docs/reports/leo-working-state-20260709/working-leo-execution-plan-current.md`
`docs/reports/leo-working-state-20260709/working-leo-open-ended-benchmark-spec.json`
`scripts/kb_cross_surface_resolution_packet.py`
`scripts/working_leo_open_ended_benchmark.py`
`tests/test_kb_cross_surface_resolution_packet.py`
`tests/test_working_leo_open_ended_benchmark.py`
* feat(kb): apply-worker to auto-land approved proposals (stage 2 automation)
Event-driven worker that turns a HUMAN-approved kb_stage proposal into canonical
state, so an approval in Telegram surfaces in Leo's identity without Leo applying
its own work.
- Fires only on status='approved' (never pending_review) -> proposer != applier
holds; the human approval stays the trigger. No auto-approve anywhere.
- Reuses scripts/apply_proposal.py verbatim as the sole apply path (same txn,
rowcount=1 guard, FK stamp). Connects as the narrow kb_apply role, never
superuser, never inside the hermes harness.
- Render hook (--render-cmd / KB_APPLY_RENDER_CMD) is inert until the SOUL
renderer (PR2) is deployed; applying still works, rendered SOUL just lags.
- Ships INERT: report-only unless --enable / KB_APPLY_WORKER_ENABLED=1. systemd
oneshot service + 5min timer, both shipped disabled.
- 10 unit tests; candidate query validated read-only vs prod (0 applyable today).
* fix(kb): apply-worker --max-per-tick cap + poison-pill retry ceiling
Fixer draft-exit items:
- --max-per-tick=1 (default): an enabled worker lands applies one-at-a-time
and observably instead of draining the whole approved queue in one tick.
- --max-attempts=3 ceiling with a persisted failure-count state file: a
deterministically-failing approved proposal is treated as a poison pill and
skipped after N consecutive failures, instead of retrying every tick forever.
State persists on disk because the worker runs oneshot per timer tick.
Both are inert until the worker is enabled; it still ships disabled.
* feat(kb): apply_proposal engine to land approved proposals into canonical
Stage 2 of the KB apply pipeline (approve -> APPLY -> render -> surface).
Turns an approved kb_stage.kb_proposals row into canonical public.* rows and
flips the ledger to 'applied' in one verified transaction.
- Connects as the narrow kb_apply role (never superuser): writes only
strategies, strategy_nodes, claim_evidence, claim_edges + kb_proposals ledger.
Enforces "agents propose, do not self-apply" at the DB boundary.
- Per-type handlers: revise_strategy (versioned strategy + node replace),
add_edge, attach_evidence (requires existing source_id; source minting is
intentionally out of scope for kb_apply's grants).
- Strict apply_payload contract (v1); freeform eval packets are normalized
upstream, not applied directly.
- --dry-run prints exact SQL; idempotent (refuses non-approved / already-applied);
transactional with an in-txn DO-block invariant check that rolls back on failure.
- Unit tests cover SQL builders, validation, dispatch, and status guards.
* fix(kb): rowcount=1 apply guard + real applied_by FK stamp
Closes the three draft-exit review items on the apply engine:
- Ledger flip now runs in a DO block asserting exactly one 'approved'
row moved to 'applied' (GET DIAGNOSTICS row_count). Closes the
concurrent double-apply race — load_proposal (read) and the flip
(write) are separate statements, so a row lock cannot span them; only
one concurrent apply can match status='approved', so rowcount=1 is the
authoritative guard. A loser RAISEs and the whole txn rolls back.
- applied_by_agent_id is stamped as a real FK resolved from public.agents
by handle, defaulting to the kb-apply service agent — no more NULL FK,
no backfill needed.
- scripts/kb_apply_prereqs.sql: one-time superuser bootstrap — inserts the
kb-apply service-agent row (kb_apply never gets INSERT on agents), grants
kb_apply SELECT on public.agents, and ensures the one-active-strategy
unique index (idempotent; already present on prod).
18/18 unit tests pass.
* fix(kb): hard-resolve applied_by handle, RAISE on NULL FK
Resolve applied_by into a variable and assert NOT NULL before the ledger
flip, instead of an inline subselect that silently stamps a NULL
applied_by_agent_id on an unresolved handle. Since the FK is ON DELETE SET
NULL, a bad handle (typo/unseeded agent) was a legal silent NULL -- the
perpetually-NULL FK we eliminated. Unresolved handle now hard-fails ->
rollback. Non-default --applied-by (operator, future drafters) is the path
that goes through the lookup and could strand NULL.
* Wire Leo Telegram x402 smart research
* Suppress token-bearing Telegram HTTP logs
* Keep Telegram typing visible during Leo proxy calls
* Allow Leo Telegram social research spend cap
* Route contextual Leo research prompts to smart research
* Generalize Leo smart research intent routing
* Resume Leo smart research from paid work orders
- Encode transcript requirements for model discovery and Pentagon boundary
- Add KB read/propose skill for Hermes, OpenClaw, and Claude-style agents
- Extend LLM contract checks; verify with 422-test suite
`.agents/skills/living-ip-kb-interop/SKILL.md`
`.agents/skills/nousresearch-hermes-agent/SKILL.md`
`.agents/skills/openclaw-agent/SKILL.md`
`docs/llm-refinement-decision-engine.md`
`scripts/check_llm_refinement_contract.py`
- Define Rio and Theseus as economics and model-integrity evaluators
- Add DB, Hermes, and OpenClaw skills with no-secret defaults
- Gate CI on LLM refinement contracts; verify with 422-test suite
`.agents/skills/decision-engine-refinement/SKILL.md`
`.agents/skills/nousresearch-hermes-agent/SKILL.md`
`.agents/skills/openclaw-agent/SKILL.md`
`.agents/skills/teleo-db-operator/SKILL.md`
`.crabbox.yaml`
`.github/workflows/ci.yml`
`docs/llm-refinement-decision-engine.md`
`scripts/check_llm_refinement_contract.py`
reweave.py and ingestion run as the operator Forgejo token, so the prior
opener-based classifier set submitted_by=m3taversal for every system
maintenance PR. backfill_submitted_by.py never overrides non-NULL rows,
so this misattribution accumulated: ~2,748 reweave/ingestion PRs and
~3,706 <agent>/ research/entity PRs were credited to the operator on
the leaderboard and contribution_events table.
Two parts:
1. lib/merge.py: at PR discovery, classify by branch prefix first.
reweave/, ingestion/ -> submitted_by = 'pipeline'
<agent>/ (per _AGENT_NAMES) -> submitted_by = '<agent>'
otherwise human -> submitted_by = author.lower()
otherwise pipeline -> submitted_by = None
(extract.py sets from proposed_by)
Origin flag updated so domain detection and priority still fire for
branch-classified pipeline PRs. Human PRs lowercased to maintain the
canonical-handle contract enforced in PR #9.
2. scripts/reattribute-by-branch-prefix.py: historical cleanup.
Per affected PR (atomic):
- UPDATE prs.submitted_by -> target
- UPDATE sources.submitted_by where source_path matches
- UPDATE contribution_events handle ('m3taversal',role='author')
-> target, kind='agent'. Collision (target already has author
event for PR) deletes the m3ta row; target wins.
Scope is deliberately conservative: extract/ branches stay attributed
to m3taversal because proposed_by-missing legitimately defaults to the
operator (telegram drops). Only reweave/, ingestion/, and <agent>/.
Dry-run shows 6,454 PRs + 284 events to move. Pre-flight collision
query returns 0; pre-flight kind check confirms m3ta has only role=author
events on this set (no challenger/synthesizer/evaluator).
Idempotent. Dry-run by default. Run with --apply after deploy + DB
snapshot.
Companion / write-side fix to fix/activity-feed-canonical-handle.
The activity-feed canonicalization was a read-side guard. The bug at the
source is that extract.py and two backfill scripts write decorated
strings (Vida (self-directed), pipeline (reweave), @m3taversal) into
prs.submitted_by and sources.submitted_by. Downstream readers
(lib.contributor.insert_contribution_event, scripts/scoring_digest,
diagnostics/activity_feed_api) all strip the decorator on read — but
anything that reads the column verbatim (like /api/activity-feed before
the read-side fix) 404s on /contributors/{decorated-handle}.
Stop writing the decorator. The self-directed signal is already carried
by intake_tier == research-task plus the prs.agent column; the suffix
is redundant string noise that costs us correctness at every consumer
that forgets to strip.
Changes:
- lib/extract.py:690 — write canonical handle via attribution.normalize_handle.
Direct elif for intake_tier == research-task now stores just agent_name.
@m3taversal -> m3taversal.
- diagnostics/backfill_submitted_by.py — same fix in two branches plus
the reweave branch (pipeline (reweave) -> pipeline).
- scripts/backfill-research-session-attribution.py — UPDATE prs sets
agent handle alone, no suffix. Docstring + log line updated.
- scripts/normalize-submitted-by.py (new) — one-time backfill that
canonicalizes existing prs.submitted_by and sources.submitted_by rows.
Strips trailing parenthetical decorators, lowercases, drops @. Defaults
to dry-run; --apply to commit. Skips rows that would normalize to
invalid handles (no garbage falls through silently).
Dry-run against live pipeline.db:
prs: 3008 rows need normalization (clean mappings, 0 invalid)
sources: 730 rows need normalization (clean mappings, 0 invalid)
Total: 3738 rows. All map to existing handle column values.
After this lands + auto-deploys, the operator should run
python3 scripts/normalize-submitted-by.py --apply
once to clean historical rows. The read-side canonicalization in
diagnostics/activity_feed_api.py (fix/activity-feed-canonical-handle)
becomes redundant defense-in-depth instead of load-bearing.
No KB writes.
