Generate ready-not-executed operator runbook for the integrated apply packet set.
Extend Cory-style benchmark with no-context direct-claim prompts and expected follow-ups.
Refresh retained reports and tests for the 20-prompt mixed-evidence score.
- Run OE-01 through OE-05 live Telegram read-only benchmark coverage
- Retain full-suite scoring artifacts and no-mutation safety readback
- Broaden scorer tests for live Leo wording without weakening claim ceilings
Files:
- Rescore retained live Telegram replies with partial versus full coverage labels
- Retain OE-01 benchmark score artifacts without mutating Leo or production DB
- Update Working Leo truth docs and tests so partial proof is not overclaimed
`docs/reports/leo-working-state-20260709/current-truth-index.md`
`docs/reports/leo-working-state-20260709/telegram-live-open-ended-benchmark-score-current.json`
`docs/reports/leo-working-state-20260709/telegram-live-open-ended-benchmark-score-current.md`
`docs/reports/leo-working-state-20260709/working-leo-current-state-20260709.md`
`docs/reports/leo-working-state-20260709/working-leo-execution-plan-current.md`
`scripts/working_leo_open_ended_benchmark.py`
`tests/test_working_leo_open_ended_benchmark.py`
- Create guarded Rio strategy nodes and anchors after the mapped base packet
- Prove clone apply and rollback while production rows and Leo runtime stay unchanged
- Retain packet docs, state image, and focused tests for future apply review
`docs/reports/leo-working-state-20260709/current-truth-index.md`
`docs/reports/leo-working-state-20260709/leo-db-state-23-rio-strategy-context.svg`
`docs/reports/leo-working-state-20260709/rio-strategy-apply-authorized-commit.sql`
`docs/reports/leo-working-state-20260709/rio-strategy-apply-rollback-rehearsal.sql`
`docs/reports/leo-working-state-20260709/rio-strategy-clone-rehearsal-current.log`
`docs/reports/leo-working-state-20260709/rio-strategy-context-current.md`
`docs/reports/leo-working-state-20260709/rio-strategy-delete-rollback.sql`
`docs/reports/leo-working-state-20260709/rio-strategy-packet.json`
`docs/reports/leo-working-state-20260709/rio-strategy-postflight.sql`
`docs/reports/leo-working-state-20260709/rio-strategy-preflight.sql`
`docs/reports/leo-working-state-20260709/working-leo-current-state-20260709.md`
`docs/reports/leo-working-state-20260709/working-leo-execution-plan-current.md`
`scripts/kb_rio_strategy_context_packet.py`
`tests/test_kb_rio_strategy_context_packet.py`
- Add broad Cory-style outcome scenarios to the open-ended benchmark so regular-use ambiguity is tested, not only exact ID canaries.
- Add a clone-proven cross-surface strategy anchor packet that updates the mapped rich proposal anchor only after Claim D exists.
- Retain VPS Docker DB, claim/body metadata, and cross-surface proof reports with ruff plus 37 focused tests passing.
`docs/reports/leo-working-state-20260709/cory-expected-working-leo-outcomes-20260709.md`
`docs/reports/leo-working-state-20260709/cross-surface-apply-authorized-commit.sql`
`docs/reports/leo-working-state-20260709/cross-surface-clone-rehearsal-current.log`
`docs/reports/leo-working-state-20260709/cross-surface-clone-rehearsal-failed-source-id-constraint.log`
`docs/reports/leo-working-state-20260709/cross-surface-delete-rollback.sql`
`docs/reports/leo-working-state-20260709/cross-surface-packet.json`
`docs/reports/leo-working-state-20260709/cross-surface-postflight.sql`
`docs/reports/leo-working-state-20260709/cross-surface-preflight.sql`
`docs/reports/leo-working-state-20260709/cross-surface-resolution-current.md`
`docs/reports/leo-working-state-20260709/current-truth-index.md`
`docs/reports/leo-working-state-20260709/leo-db-state-17-cory-outcome-benchmark.svg`
`docs/reports/leo-working-state-20260709/leo-db-state-18-db-vs-workspace.svg`
`docs/reports/leo-working-state-20260709/leo-db-state-19-claims-body-metadata.svg`
`docs/reports/leo-working-state-20260709/leo-db-state-20-cross-surface-anchor.svg`
`docs/reports/leo-working-state-20260709/working-leo-current-state-20260709.md`
`docs/reports/leo-working-state-20260709/working-leo-definition-20260709.md`
`docs/reports/leo-working-state-20260709/working-leo-execution-plan-current.md`
`docs/reports/leo-working-state-20260709/working-leo-open-ended-benchmark-spec.json`
`scripts/kb_cross_surface_resolution_packet.py`
`scripts/working_leo_open_ended_benchmark.py`
`tests/test_kb_cross_surface_resolution_packet.py`
`tests/test_working_leo_open_ended_benchmark.py`
Render KB claim links in leoclean replies.
Local evidence:
- 13 focused tests passed.
- py_compile passed for Telegram bot, both leoclean KB bridges, and KB claim routes.
- ruff F,E9 passed for touched files.
GitHub checks were queued at merge time, not failing.
* feat(kb): apply-worker to auto-land approved proposals (stage 2 automation)
Event-driven worker that turns a HUMAN-approved kb_stage proposal into canonical
state, so an approval in Telegram surfaces in Leo's identity without Leo applying
its own work.
- Fires only on status='approved' (never pending_review) -> proposer != applier
holds; the human approval stays the trigger. No auto-approve anywhere.
- Reuses scripts/apply_proposal.py verbatim as the sole apply path (same txn,
rowcount=1 guard, FK stamp). Connects as the narrow kb_apply role, never
superuser, never inside the hermes harness.
- Render hook (--render-cmd / KB_APPLY_RENDER_CMD) is inert until the SOUL
renderer (PR2) is deployed; applying still works, rendered SOUL just lags.
- Ships INERT: report-only unless --enable / KB_APPLY_WORKER_ENABLED=1. systemd
oneshot service + 5min timer, both shipped disabled.
- 10 unit tests; candidate query validated read-only vs prod (0 applyable today).
* fix(kb): apply-worker --max-per-tick cap + poison-pill retry ceiling
Fixer draft-exit items:
- --max-per-tick=1 (default): an enabled worker lands applies one-at-a-time
and observably instead of draining the whole approved queue in one tick.
- --max-attempts=3 ceiling with a persisted failure-count state file: a
deterministically-failing approved proposal is treated as a poison pill and
skipped after N consecutive failures, instead of retrying every tick forever.
State persists on disk because the worker runs oneshot per timer tick.
Both are inert until the worker is enabled; it still ships disabled.
* feat(kb): apply_proposal engine to land approved proposals into canonical
Stage 2 of the KB apply pipeline (approve -> APPLY -> render -> surface).
Turns an approved kb_stage.kb_proposals row into canonical public.* rows and
flips the ledger to 'applied' in one verified transaction.
- Connects as the narrow kb_apply role (never superuser): writes only
strategies, strategy_nodes, claim_evidence, claim_edges + kb_proposals ledger.
Enforces "agents propose, do not self-apply" at the DB boundary.
- Per-type handlers: revise_strategy (versioned strategy + node replace),
add_edge, attach_evidence (requires existing source_id; source minting is
intentionally out of scope for kb_apply's grants).
- Strict apply_payload contract (v1); freeform eval packets are normalized
upstream, not applied directly.
- --dry-run prints exact SQL; idempotent (refuses non-approved / already-applied);
transactional with an in-txn DO-block invariant check that rolls back on failure.
- Unit tests cover SQL builders, validation, dispatch, and status guards.
* fix(kb): rowcount=1 apply guard + real applied_by FK stamp
Closes the three draft-exit review items on the apply engine:
- Ledger flip now runs in a DO block asserting exactly one 'approved'
row moved to 'applied' (GET DIAGNOSTICS row_count). Closes the
concurrent double-apply race — load_proposal (read) and the flip
(write) are separate statements, so a row lock cannot span them; only
one concurrent apply can match status='approved', so rowcount=1 is the
authoritative guard. A loser RAISEs and the whole txn rolls back.
- applied_by_agent_id is stamped as a real FK resolved from public.agents
by handle, defaulting to the kb-apply service agent — no more NULL FK,
no backfill needed.
- scripts/kb_apply_prereqs.sql: one-time superuser bootstrap — inserts the
kb-apply service-agent row (kb_apply never gets INSERT on agents), grants
kb_apply SELECT on public.agents, and ensures the one-active-strategy
unique index (idempotent; already present on prod).
18/18 unit tests pass.
* fix(kb): hard-resolve applied_by handle, RAISE on NULL FK
Resolve applied_by into a variable and assert NOT NULL before the ledger
flip, instead of an inline subselect that silently stamps a NULL
applied_by_agent_id on an unresolved handle. Since the FK is ON DELETE SET
NULL, a bad handle (typo/unseeded agent) was a legal silent NULL -- the
perpetually-NULL FK we eliminated. Unresolved handle now hard-fails ->
rollback. Non-default --applied-by (operator, future drafters) is the path
that goes through the lookup and could strand NULL.
Adds graph schema prerequisite plus research-eval schema/docs/tests for Leo tool-use benchmarks and x402 research telemetry. Validated by full local pytest and green CI.
* Wire Leo Telegram x402 smart research
* Suppress token-bearing Telegram HTTP logs
* Keep Telegram typing visible during Leo proxy calls
* Allow Leo Telegram social research spend cap
* Route contextual Leo research prompts to smart research
* Generalize Leo smart research intent routing
* Resume Leo smart research from paid work orders
3 nits from review of d60b6f8 + Q4 ask:
1. test_window_24h_only_today: replace always-true assertion with
concrete `assert handles == ["carol"]`. Push alice's most-recent
event from -1 days to -2 days to eliminate fixture-vs-query
microsecond drift on the 24h boundary.
2. _call helper: asyncio.get_event_loop().run_until_complete →
asyncio.run (deprecation in 3.12, raises in some 3.14 contexts).
3. test_invalid_limit_falls_to_default: dead first call removed,
misleading "7 entries" comment now matches assertion.
Q4: scripts/reset-m3taversal-sourcer.py captures the surgical
UPDATE we ran on VPS as a reviewable artifact. Idempotent (no-op
on already-reset rows), audit_log entry per run. Ganymede's point:
DB mutations should leave a code paper trail, not just an audit
row whose origin lives only in the executor's memory.
30/30 tests pass on VPS hermes venv (aiohttp 3.13.5, py 3.11.15).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds tests/test_leaderboard.py — 30 cases against
diagnostics/leaderboard_routes.py. Two reasons:
(1) Zero coverage on an endpoint Argus + Oberon are about to consume
for the May 5 hackathon UI. Two bugs slipped through this morning
(404 wiring missing in app.py; AND-prefix SQL syntax error on
rolling-window). Tests prevent regression.
(2) Tests serve as living documentation for Oberon's frontend
integration — each test names a contract guarantee
(test_left_join_handles_missing_contributors_row,
test_composed_window_kind_domain, test_role_breakdown_present).
Coverage:
- _parse_window unit tests (10): all_time, Nd, Nh, caps, garbage,
case-normalization, and explicit no-AND-prefix assertion
- handle_leaderboard integration (18): every kind value, every
window family, domain filter, composed filters, limit + has_more,
invalid-input fallback, role breakdown shape, empty-window shape,
LEFT JOIN COALESCE for handles missing from contributors
- 2 contract assertions: LEADERBOARD_PUBLIC_PATHS membership +
KIND_VALUES set
Run: 30/30 pass on VPS hermes venv (aiohttp 3.13.5, pytest 9.0.2).
Skips clean locally without aiohttp via pytest.importorskip.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Move tests from /tmp into the proper test suite. 22 cases covering:
- Leo gotcha: extract/* + commit_type=enrich/challenge classifies by
commit_type, not branch prefix (same pattern as the contributor-role
wiring fix)
- Reweave priority: branch.startswith('reweave/') wins over
_MAINTENANCE_COMMIT_TYPES — nightly reweave PRs classify as enrich,
not infra. Locks in the bifurcation against future priority refactors
- Full NON_MERGED_STATUS_TO_OPERATION coverage: open, approved, closed,
conflict, validating, reviewing, merging, zombie
- Knowledge-producing commit_types (research, entity) → new
- Maintenance commit_types (fix, pipeline) → infra
- Defensive: null inputs, unknown status
aiohttp imported at module load — file uses pytest.importorskip so it
runs cleanly in any environment with aiohttp installed and skips gracefully
otherwise. sys.path inject for diagnostics/ since it isn't packaged.
Reviewed-by: Ganymede
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
