Generate ready-not-executed operator runbook for the integrated apply packet set.
Extend Cory-style benchmark with no-context direct-claim prompts and expected follow-ups.
Refresh retained reports and tests for the 20-prompt mixed-evidence score.
- Run OE-01 through OE-05 live Telegram read-only benchmark coverage
- Retain full-suite scoring artifacts and no-mutation safety readback
- Broaden scorer tests for live Leo wording without weakening claim ceilings
Files:
- Rescore retained live Telegram replies with partial versus full coverage labels
- Retain OE-01 benchmark score artifacts without mutating Leo or production DB
- Update Working Leo truth docs and tests so partial proof is not overclaimed
`docs/reports/leo-working-state-20260709/current-truth-index.md`
`docs/reports/leo-working-state-20260709/telegram-live-open-ended-benchmark-score-current.json`
`docs/reports/leo-working-state-20260709/telegram-live-open-ended-benchmark-score-current.md`
`docs/reports/leo-working-state-20260709/working-leo-current-state-20260709.md`
`docs/reports/leo-working-state-20260709/working-leo-execution-plan-current.md`
`scripts/working_leo_open_ended_benchmark.py`
`tests/test_working_leo_open_ended_benchmark.py`
- Create guarded Rio strategy nodes and anchors after the mapped base packet
- Prove clone apply and rollback while production rows and Leo runtime stay unchanged
- Retain packet docs, state image, and focused tests for future apply review
`docs/reports/leo-working-state-20260709/current-truth-index.md`
`docs/reports/leo-working-state-20260709/leo-db-state-23-rio-strategy-context.svg`
`docs/reports/leo-working-state-20260709/rio-strategy-apply-authorized-commit.sql`
`docs/reports/leo-working-state-20260709/rio-strategy-apply-rollback-rehearsal.sql`
`docs/reports/leo-working-state-20260709/rio-strategy-clone-rehearsal-current.log`
`docs/reports/leo-working-state-20260709/rio-strategy-context-current.md`
`docs/reports/leo-working-state-20260709/rio-strategy-delete-rollback.sql`
`docs/reports/leo-working-state-20260709/rio-strategy-packet.json`
`docs/reports/leo-working-state-20260709/rio-strategy-postflight.sql`
`docs/reports/leo-working-state-20260709/rio-strategy-preflight.sql`
`docs/reports/leo-working-state-20260709/working-leo-current-state-20260709.md`
`docs/reports/leo-working-state-20260709/working-leo-execution-plan-current.md`
`scripts/kb_rio_strategy_context_packet.py`
`tests/test_kb_rio_strategy_context_packet.py`
- Add broad Cory-style outcome scenarios to the open-ended benchmark so regular-use ambiguity is tested, not only exact ID canaries.
- Add a clone-proven cross-surface strategy anchor packet that updates the mapped rich proposal anchor only after Claim D exists.
- Retain VPS Docker DB, claim/body metadata, and cross-surface proof reports with ruff plus 37 focused tests passing.
`docs/reports/leo-working-state-20260709/cory-expected-working-leo-outcomes-20260709.md`
`docs/reports/leo-working-state-20260709/cross-surface-apply-authorized-commit.sql`
`docs/reports/leo-working-state-20260709/cross-surface-clone-rehearsal-current.log`
`docs/reports/leo-working-state-20260709/cross-surface-clone-rehearsal-failed-source-id-constraint.log`
`docs/reports/leo-working-state-20260709/cross-surface-delete-rollback.sql`
`docs/reports/leo-working-state-20260709/cross-surface-packet.json`
`docs/reports/leo-working-state-20260709/cross-surface-postflight.sql`
`docs/reports/leo-working-state-20260709/cross-surface-preflight.sql`
`docs/reports/leo-working-state-20260709/cross-surface-resolution-current.md`
`docs/reports/leo-working-state-20260709/current-truth-index.md`
`docs/reports/leo-working-state-20260709/leo-db-state-17-cory-outcome-benchmark.svg`
`docs/reports/leo-working-state-20260709/leo-db-state-18-db-vs-workspace.svg`
`docs/reports/leo-working-state-20260709/leo-db-state-19-claims-body-metadata.svg`
`docs/reports/leo-working-state-20260709/leo-db-state-20-cross-surface-anchor.svg`
`docs/reports/leo-working-state-20260709/working-leo-current-state-20260709.md`
`docs/reports/leo-working-state-20260709/working-leo-definition-20260709.md`
`docs/reports/leo-working-state-20260709/working-leo-execution-plan-current.md`
`docs/reports/leo-working-state-20260709/working-leo-open-ended-benchmark-spec.json`
`scripts/kb_cross_surface_resolution_packet.py`
`scripts/working_leo_open_ended_benchmark.py`
`tests/test_kb_cross_surface_resolution_packet.py`
`tests/test_working_leo_open_ended_benchmark.py`
Render KB claim links in leoclean replies.
Local evidence:
- 13 focused tests passed.
- py_compile passed for Telegram bot, both leoclean KB bridges, and KB claim routes.
- ruff F,E9 passed for touched files.
GitHub checks were queued at merge time, not failing.
* feat(kb): apply-worker to auto-land approved proposals (stage 2 automation)
Event-driven worker that turns a HUMAN-approved kb_stage proposal into canonical
state, so an approval in Telegram surfaces in Leo's identity without Leo applying
its own work.
- Fires only on status='approved' (never pending_review) -> proposer != applier
holds; the human approval stays the trigger. No auto-approve anywhere.
- Reuses scripts/apply_proposal.py verbatim as the sole apply path (same txn,
rowcount=1 guard, FK stamp). Connects as the narrow kb_apply role, never
superuser, never inside the hermes harness.
- Render hook (--render-cmd / KB_APPLY_RENDER_CMD) is inert until the SOUL
renderer (PR2) is deployed; applying still works, rendered SOUL just lags.
- Ships INERT: report-only unless --enable / KB_APPLY_WORKER_ENABLED=1. systemd
oneshot service + 5min timer, both shipped disabled.
- 10 unit tests; candidate query validated read-only vs prod (0 applyable today).
* fix(kb): apply-worker --max-per-tick cap + poison-pill retry ceiling
Fixer draft-exit items:
- --max-per-tick=1 (default): an enabled worker lands applies one-at-a-time
and observably instead of draining the whole approved queue in one tick.
- --max-attempts=3 ceiling with a persisted failure-count state file: a
deterministically-failing approved proposal is treated as a poison pill and
skipped after N consecutive failures, instead of retrying every tick forever.
State persists on disk because the worker runs oneshot per timer tick.
Both are inert until the worker is enabled; it still ships disabled.
* feat(kb): apply_proposal engine to land approved proposals into canonical
Stage 2 of the KB apply pipeline (approve -> APPLY -> render -> surface).
Turns an approved kb_stage.kb_proposals row into canonical public.* rows and
flips the ledger to 'applied' in one verified transaction.
- Connects as the narrow kb_apply role (never superuser): writes only
strategies, strategy_nodes, claim_evidence, claim_edges + kb_proposals ledger.
Enforces "agents propose, do not self-apply" at the DB boundary.
- Per-type handlers: revise_strategy (versioned strategy + node replace),
add_edge, attach_evidence (requires existing source_id; source minting is
intentionally out of scope for kb_apply's grants).
- Strict apply_payload contract (v1); freeform eval packets are normalized
upstream, not applied directly.
- --dry-run prints exact SQL; idempotent (refuses non-approved / already-applied);
transactional with an in-txn DO-block invariant check that rolls back on failure.
- Unit tests cover SQL builders, validation, dispatch, and status guards.
* fix(kb): rowcount=1 apply guard + real applied_by FK stamp
Closes the three draft-exit review items on the apply engine:
- Ledger flip now runs in a DO block asserting exactly one 'approved'
row moved to 'applied' (GET DIAGNOSTICS row_count). Closes the
concurrent double-apply race — load_proposal (read) and the flip
(write) are separate statements, so a row lock cannot span them; only
one concurrent apply can match status='approved', so rowcount=1 is the
authoritative guard. A loser RAISEs and the whole txn rolls back.
- applied_by_agent_id is stamped as a real FK resolved from public.agents
by handle, defaulting to the kb-apply service agent — no more NULL FK,
no backfill needed.
- scripts/kb_apply_prereqs.sql: one-time superuser bootstrap — inserts the
kb-apply service-agent row (kb_apply never gets INSERT on agents), grants
kb_apply SELECT on public.agents, and ensures the one-active-strategy
unique index (idempotent; already present on prod).
18/18 unit tests pass.
* fix(kb): hard-resolve applied_by handle, RAISE on NULL FK
Resolve applied_by into a variable and assert NOT NULL before the ledger
flip, instead of an inline subselect that silently stamps a NULL
applied_by_agent_id on an unresolved handle. Since the FK is ON DELETE SET
NULL, a bad handle (typo/unseeded agent) was a legal silent NULL -- the
perpetually-NULL FK we eliminated. Unresolved handle now hard-fails ->
rollback. Non-default --applied-by (operator, future drafters) is the path
that goes through the lookup and could strand NULL.
Adds graph schema prerequisite plus research-eval schema/docs/tests for Leo tool-use benchmarks and x402 research telemetry. Validated by full local pytest and green CI.
* Wire Leo Telegram x402 smart research
* Suppress token-bearing Telegram HTTP logs
* Keep Telegram typing visible during Leo proxy calls
* Allow Leo Telegram social research spend cap
* Route contextual Leo research prompts to smart research
* Generalize Leo smart research intent routing
* Resume Leo smart research from paid work orders
3 nits from review of d60b6f8 + Q4 ask:
1. test_window_24h_only_today: replace always-true assertion with
concrete `assert handles == ["carol"]`. Push alice's most-recent
event from -1 days to -2 days to eliminate fixture-vs-query
microsecond drift on the 24h boundary.
2. _call helper: asyncio.get_event_loop().run_until_complete →
asyncio.run (deprecation in 3.12, raises in some 3.14 contexts).
3. test_invalid_limit_falls_to_default: dead first call removed,
misleading "7 entries" comment now matches assertion.
Q4: scripts/reset-m3taversal-sourcer.py captures the surgical
UPDATE we ran on VPS as a reviewable artifact. Idempotent (no-op
on already-reset rows), audit_log entry per run. Ganymede's point:
DB mutations should leave a code paper trail, not just an audit
row whose origin lives only in the executor's memory.
30/30 tests pass on VPS hermes venv (aiohttp 3.13.5, py 3.11.15).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds tests/test_leaderboard.py — 30 cases against
diagnostics/leaderboard_routes.py. Two reasons:
(1) Zero coverage on an endpoint Argus + Oberon are about to consume
for the May 5 hackathon UI. Two bugs slipped through this morning
(404 wiring missing in app.py; AND-prefix SQL syntax error on
rolling-window). Tests prevent regression.
(2) Tests serve as living documentation for Oberon's frontend
integration — each test names a contract guarantee
(test_left_join_handles_missing_contributors_row,
test_composed_window_kind_domain, test_role_breakdown_present).
Coverage:
- _parse_window unit tests (10): all_time, Nd, Nh, caps, garbage,
case-normalization, and explicit no-AND-prefix assertion
- handle_leaderboard integration (18): every kind value, every
window family, domain filter, composed filters, limit + has_more,
invalid-input fallback, role breakdown shape, empty-window shape,
LEFT JOIN COALESCE for handles missing from contributors
- 2 contract assertions: LEADERBOARD_PUBLIC_PATHS membership +
KIND_VALUES set
Run: 30/30 pass on VPS hermes venv (aiohttp 3.13.5, pytest 9.0.2).
Skips clean locally without aiohttp via pytest.importorskip.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Move tests from /tmp into the proper test suite. 22 cases covering:
- Leo gotcha: extract/* + commit_type=enrich/challenge classifies by
commit_type, not branch prefix (same pattern as the contributor-role
wiring fix)
- Reweave priority: branch.startswith('reweave/') wins over
_MAINTENANCE_COMMIT_TYPES — nightly reweave PRs classify as enrich,
not infra. Locks in the bifurcation against future priority refactors
- Full NON_MERGED_STATUS_TO_OPERATION coverage: open, approved, closed,
conflict, validating, reviewing, merging, zombie
- Knowledge-producing commit_types (research, entity) → new
- Maintenance commit_types (fix, pipeline) → infra
- Defensive: null inputs, unknown status
aiohttp imported at module load — file uses pytest.importorskip so it
runs cleanly in any environment with aiohttp installed and skips gracefully
otherwise. sys.path inject for diagnostics/ since it isn't packaged.
Reviewed-by: Ganymede
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Five tests against the real contribution_events schema (lib/db.py:181-209):
- pr-level dedup with NULL claim_path via idx_ce_unique_pr partial index
- per-claim dedup with non-NULL claim_path via idx_ce_unique_claim partial index
- pr-level and per-claim events coexist on the same pr_number
- backfill (INSERT correct + DELETE wrong) is a true no-op on replay
- replay against already-backfilled state preserves unrelated events
Schema case identified: case 2 with partial-index split solution already in
place. Two partial UNIQUE indexes target disjoint row sets (claim_path IS NULL
vs IS NOT NULL), bypassing SQLite's NULL-not-equal-NULL UNIQUE quirk.
Production replay verified: re-running backfill --apply against the live DB
returns "misattributed PRs found: 0" because the first-run UPDATE flipped the
WHERE predicate. Total contribution_events count: 3839 → 3839.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Ganymede review findings on epimetheus/contributor-attribution-fix branch:
1. BUG: record_contributor_attribution used `git diff --name-only` (all modified
files), not just added. Enrich/challenge PRs re-credited the sourcer on every
subsequent modification. Fixed: --diff-filter=A restricts to new files only.
The synthesizer/challenger/reviewer roles for enrich PRs are still credited
via the Pentagon-Agent trailer path, so this doesn't lose any correct credit.
2. WARNING: Legacy `source`-field heuristic fabricated garbage handles from
descriptive strings ("sec-interpretive-release-s7-2026-09-(march-17",
"governance---meritocratic-voting-+-futarchy"). Removed outright + added
regex handle sanity filter (`^[a-z0-9][a-z0-9_-]{0,38}$`). Applied before
every return path in parse_attribution (the nested-block early return was
previously bypassing the filter).
Dry-run impact: unique handles 83→70 (13 garbage filtered), NEW contributors
49→48, EXISTING drift rows 34→22. The filter drops rows where the literal
garbage string lives in frontmatter (Slotkin case: attribution.sourcer.handle
was written as "senator-elissa-slotkin-/-the-hill" by the buggy legacy path).
3. NIT: Aligned knowledge_prefixes in the file walker to match is_knowledge_pr
(removed entities/, convictions/). Widening those requires Cory sign-off
since is_knowledge_pr currently gates entity-only PRs out of CI.
Tests: 17 pass (added test_bad_handles_filtered, test_valid_handle_with_hyphen_passes,
updated test_legacy_source_fallback → test_legacy_source_fallback_removed).
Ganymede review — 3-message protocol msg 3 pending.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
deploy.sh was missing telegram/ and tests/ directories — code existed in
repo but never synced to VPS. Also removes hardcoded twitterapi.io key
from x-ingest.py (reads from secrets file like all other modules).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three changes:
1. Drop underscore prefixes in eval_parse.py — functions are now the public
API of the module (filter_diff, parse_verdict, classify_issues, etc.).
All 12 functions renamed, imports updated in evaluate.py and tests.
2. Extract eval_actions.py from evaluate.py — 3 async PR disposition functions:
- post_formal_approvals: submit Forgejo reviews from 2 agents
- terminate_pr: close PR, post rejection comment, requeue source
- dispose_rejected_pr: disposition logic for rejected PRs on attempt 2+
evaluate.py drops from ~1140 to 911 lines.
3. 14 new tests in test_eval_actions.py covering all three functions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Critical bug fix: close_pr now checks forgejo_api return value and
skips DB update on Forgejo failure, preventing ghost PRs (DB closed,
Forgejo open). Returns bool so callers can handle failures.
_terminate_pr checks return value — skips source requeue on failure.
stale_pr.py migrated from raw Forgejo+DB to close_pr (last raw close
transition eliminated).
eval_parse.py: 15 pure parsing functions extracted from evaluate.py
(~370 lines removed). Zero I/O, zero async, independently testable.
evaluate.py drops from ~1510 to ~1140 lines.
Tests: 295 passed (42 new eval_parse + 2 new close_pr), zero regressions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fix 4 Forgejo ghost PR bugs flagged by Ganymede:
- fixer.py GC close: DB update ran outside try/except, closing DB even on Forgejo failure
- substantive_fixer.py droppable: NO Forgejo close at all
- substantive_fixer.py auto-enrichment: DB update before Forgejo (reversed order)
- substantive_fixer.py close_and_reextract: replace manual Forgejo+DB with close_pr()
Add start_fixing() and reset_for_reeval() to pr_state.py:
- start_fixing: atomic claim + fix_attempts increment in one statement
- reset_for_reeval: clears all eval state for re-evaluation after fix
Also fixes stale line number comment in merge.py (Ganymede nit).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace 38 hand-crafted UPDATE prs SET status calls across evaluate.py
and merge.py with 7 centralized functions that enforce invariants:
- close_pr: always syncs Forgejo (opt-out for reconciliation)
- approve_pr: raises ValueError on empty domain (prevents NULL bugs)
- mark_merged: always sets merged_at, clears last_error
- mark_conflict: always increments merge_failures, sets merge_cycled
- mark_conflict_permanent: terminal conflict state
- reopen_pr: handles all reopen scenarios (transient, rejection, reeval)
- start_review: atomic claim with bool return
This eliminates the class of bugs that produced 3 incidents:
1. Domain NULL on musings bypass (7 PRs stuck, 20h zero throughput)
2. Forgejo ghost PRs (70 PRs open on Forgejo but closed in DB)
3. Merge_cycled missing on various close paths
Also fixes: 3 close paths in merge.py had DB update before Forgejo call
(reversed order). close_pr does Forgejo first, then DB.
Only remaining raw status transition: _claim_next_pr (approved→merging)
which is an atomic subquery and doesn't have invariant requirements.
20 new tests, 264 total passing, 0 regressions. Net -101 lines in
evaluate.py + merge.py.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Output gate (output_gate.py): Deterministic classifier that blocks system/pipeline
messages from reaching public outputs. Pattern-based detection of PR numbers,
deploy logs, diagnostics, infrastructure references.
Tweet queue (x_publisher.py): Submit drafts through output gate + OPSEC filter,
enter approval_queue, auto-post to X via Twitter API v2 on Cory's approval.
Pluggable approval stages (approval_stages.py): Extensible architecture where
adding a new approval stage = implementing ApprovalStage.check(). Current stages:
OutputGate (stage 0), OPSEC (stage 1), Human (stage 10). Designed for future
agent voting, multi-human approval, and decision markets.
Also syncs approvals.py from VPS to local repo (was deployed but never committed).
18 tests pass.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The skip loop only matched `- ` (no indent) but YAML list items are
commonly written as ` - item` (2-space indent). This caused old list
items to persist alongside new ones, corrupting frontmatter on merge.
Fix: consume any line starting with space or dash as part of the current
field's value block.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Two fixes from Ganymede review:
1. CRITICAL: blank line before closing --- compounded on repeat reweaves.
Body starts with \n---, so \n{body} created \n\n---. Fixed by checking
body prefix.
2. Replaced yaml.dump round-trip with _serialize_edge_fields() that splices
only edge arrays into raw frontmatter text. Non-edge fields (title,
confidence, type, quotes, flow styles) stay byte-identical to main HEAD.
_parse_yaml_frontmatter now returns 3-tuple: (dict, raw_fm_text, body).
_serialize_frontmatter takes (raw_fm_text, merged_edges_dict, body).
26 tests pass including idempotency (5x serialize), formatting preservation,
and no-blank-line regression test.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Reweave PRs modify existing files (appending YAML edges). Cherry-pick
fails ~75% when main moves between PR creation and merge.
_merge_reweave_pr() reads each changed file from both main HEAD and
branch HEAD, unions the edge arrays (order-preserving, main-first),
and writes the result. Eliminates merge conflicts structurally.
Key design decisions (Ganymede + Theseus approved):
- Order-preserving dedup: main's edges first, branch-new appended
- Superset assertion: logs warning if branch missing main edges
- Uses main's body text (reweave only touches frontmatter)
- Loud failure on parse errors (no cherry-pick fallback)
- Append-only contract: reweave adds edges, never removes
18 tests covering parse, union, serialize, superset, and full workflow.
Also fixes _is_entity path check to use Path.parts instead of string
containment, preventing false positives on paths like "domains/entities-overview/".
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause: _group_into_windows never checked time gaps or chat_id.
All messages went into one stream, capped at 10 per window. 120 msgs
from one chat → 12 windows → 12 source files → 12 extraction branches.
Fix:
- Group by chat_id first (different chats = different windows always)
- Split on actual time gaps (>window_seconds between messages)
- Cap at 50 messages per window (not 10)
- Consolidate substantive windows from same chat into one source file
at triage time (one source per chat per triage cycle)
6 tests in tests/test_tg_batching.py.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Layer 1: Insertion-time dedup in openrouter-extract-v2.py — skip if source_slug
already appears in claim content.
Layer 2: Insertion-time dedup in entity_batch.py — skip if PR number already
enriched this claim.
Layer 3: Post-rebase dedup in merge.py — scan rebased files for duplicate
evidence blocks (same source reference) and remove them before force-push.
Root cause: multiple enrichment branches modify the same claim at the same
insertion point. When rebased sequentially, evidence blocks are duplicated.
(Leo: PRs #1751, #1752)
lib/dedup.py: standalone module — parses evidence headers, deduplicates by
source key, preserves trailing content (Relevant Notes, Topics sections).
9 tests covering all patterns including the real PR #1751 duplication case.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Atomic extract-and-connect (lib/connect.py):
- After extraction writes claim files, each new claim is embedded via
OpenRouter, searched against Qdrant, and top-5 neighbors (cosine > 0.55)
are added as `related` edges in the claim's frontmatter
- Edges written on NEW claim only — avoids merge conflicts
- Cross-domain connections enabled, non-fatal on Qdrant failure
- Wired into openrouter-extract-v2.py post-extraction step
Stale PR monitor (lib/stale_pr.py):
- Every watchdog cycle checks open extract/* PRs
- If open >30 min AND 0 claim files → auto-close with comment
- After 2 stale closures → marks source as extraction_failed
- Wired into watchdog.py as check #6
Response audit system:
- response_audit table (migration v8), persistent audit conn in bot.py
- 90-day retention cleanup, tool_calls JSON column
- Confidence tag stripping, systemd ReadWritePaths for pipeline.db
Supporting infrastructure:
- reweave.py: nightly edge reconnection for orphan claims
- reconcile-sources.py: source status reconciliation
- backfill-domains.py: domain classification backfill
- ops/reconcile-source-status.sh: operational reconciliation script
- Attribution improvements, post-extract enrichments, merge improvements
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
