Generate ready-not-executed operator runbook for the integrated apply packet set.
Extend Cory-style benchmark with no-context direct-claim prompts and expected follow-ups.
Refresh retained reports and tests for the 20-prompt mixed-evidence score.
- Run OE-01 through OE-05 live Telegram read-only benchmark coverage
- Retain full-suite scoring artifacts and no-mutation safety readback
- Broaden scorer tests for live Leo wording without weakening claim ceilings
Files:
- Rescore retained live Telegram replies with partial versus full coverage labels
- Retain OE-01 benchmark score artifacts without mutating Leo or production DB
- Update Working Leo truth docs and tests so partial proof is not overclaimed
`docs/reports/leo-working-state-20260709/current-truth-index.md`
`docs/reports/leo-working-state-20260709/telegram-live-open-ended-benchmark-score-current.json`
`docs/reports/leo-working-state-20260709/telegram-live-open-ended-benchmark-score-current.md`
`docs/reports/leo-working-state-20260709/working-leo-current-state-20260709.md`
`docs/reports/leo-working-state-20260709/working-leo-execution-plan-current.md`
`scripts/working_leo_open_ended_benchmark.py`
`tests/test_working_leo_open_ended_benchmark.py`
- Create guarded Rio strategy nodes and anchors after the mapped base packet
- Prove clone apply and rollback while production rows and Leo runtime stay unchanged
- Retain packet docs, state image, and focused tests for future apply review
`docs/reports/leo-working-state-20260709/current-truth-index.md`
`docs/reports/leo-working-state-20260709/leo-db-state-23-rio-strategy-context.svg`
`docs/reports/leo-working-state-20260709/rio-strategy-apply-authorized-commit.sql`
`docs/reports/leo-working-state-20260709/rio-strategy-apply-rollback-rehearsal.sql`
`docs/reports/leo-working-state-20260709/rio-strategy-clone-rehearsal-current.log`
`docs/reports/leo-working-state-20260709/rio-strategy-context-current.md`
`docs/reports/leo-working-state-20260709/rio-strategy-delete-rollback.sql`
`docs/reports/leo-working-state-20260709/rio-strategy-packet.json`
`docs/reports/leo-working-state-20260709/rio-strategy-postflight.sql`
`docs/reports/leo-working-state-20260709/rio-strategy-preflight.sql`
`docs/reports/leo-working-state-20260709/working-leo-current-state-20260709.md`
`docs/reports/leo-working-state-20260709/working-leo-execution-plan-current.md`
`scripts/kb_rio_strategy_context_packet.py`
`tests/test_kb_rio_strategy_context_packet.py`
- Add broad Cory-style outcome scenarios to the open-ended benchmark so regular-use ambiguity is tested, not only exact ID canaries.
- Add a clone-proven cross-surface strategy anchor packet that updates the mapped rich proposal anchor only after Claim D exists.
- Retain VPS Docker DB, claim/body metadata, and cross-surface proof reports with ruff plus 37 focused tests passing.
`docs/reports/leo-working-state-20260709/cory-expected-working-leo-outcomes-20260709.md`
`docs/reports/leo-working-state-20260709/cross-surface-apply-authorized-commit.sql`
`docs/reports/leo-working-state-20260709/cross-surface-clone-rehearsal-current.log`
`docs/reports/leo-working-state-20260709/cross-surface-clone-rehearsal-failed-source-id-constraint.log`
`docs/reports/leo-working-state-20260709/cross-surface-delete-rollback.sql`
`docs/reports/leo-working-state-20260709/cross-surface-packet.json`
`docs/reports/leo-working-state-20260709/cross-surface-postflight.sql`
`docs/reports/leo-working-state-20260709/cross-surface-preflight.sql`
`docs/reports/leo-working-state-20260709/cross-surface-resolution-current.md`
`docs/reports/leo-working-state-20260709/current-truth-index.md`
`docs/reports/leo-working-state-20260709/leo-db-state-17-cory-outcome-benchmark.svg`
`docs/reports/leo-working-state-20260709/leo-db-state-18-db-vs-workspace.svg`
`docs/reports/leo-working-state-20260709/leo-db-state-19-claims-body-metadata.svg`
`docs/reports/leo-working-state-20260709/leo-db-state-20-cross-surface-anchor.svg`
`docs/reports/leo-working-state-20260709/working-leo-current-state-20260709.md`
`docs/reports/leo-working-state-20260709/working-leo-definition-20260709.md`
`docs/reports/leo-working-state-20260709/working-leo-execution-plan-current.md`
`docs/reports/leo-working-state-20260709/working-leo-open-ended-benchmark-spec.json`
`scripts/kb_cross_surface_resolution_packet.py`
`scripts/working_leo_open_ended_benchmark.py`
`tests/test_kb_cross_surface_resolution_packet.py`
`tests/test_working_leo_open_ended_benchmark.py`
* feat(kb): apply-worker to auto-land approved proposals (stage 2 automation)
Event-driven worker that turns a HUMAN-approved kb_stage proposal into canonical
state, so an approval in Telegram surfaces in Leo's identity without Leo applying
its own work.
- Fires only on status='approved' (never pending_review) -> proposer != applier
holds; the human approval stays the trigger. No auto-approve anywhere.
- Reuses scripts/apply_proposal.py verbatim as the sole apply path (same txn,
rowcount=1 guard, FK stamp). Connects as the narrow kb_apply role, never
superuser, never inside the hermes harness.
- Render hook (--render-cmd / KB_APPLY_RENDER_CMD) is inert until the SOUL
renderer (PR2) is deployed; applying still works, rendered SOUL just lags.
- Ships INERT: report-only unless --enable / KB_APPLY_WORKER_ENABLED=1. systemd
oneshot service + 5min timer, both shipped disabled.
- 10 unit tests; candidate query validated read-only vs prod (0 applyable today).
* fix(kb): apply-worker --max-per-tick cap + poison-pill retry ceiling
Fixer draft-exit items:
- --max-per-tick=1 (default): an enabled worker lands applies one-at-a-time
and observably instead of draining the whole approved queue in one tick.
- --max-attempts=3 ceiling with a persisted failure-count state file: a
deterministically-failing approved proposal is treated as a poison pill and
skipped after N consecutive failures, instead of retrying every tick forever.
State persists on disk because the worker runs oneshot per timer tick.
Both are inert until the worker is enabled; it still ships disabled.
* feat(kb): apply_proposal engine to land approved proposals into canonical
Stage 2 of the KB apply pipeline (approve -> APPLY -> render -> surface).
Turns an approved kb_stage.kb_proposals row into canonical public.* rows and
flips the ledger to 'applied' in one verified transaction.
- Connects as the narrow kb_apply role (never superuser): writes only
strategies, strategy_nodes, claim_evidence, claim_edges + kb_proposals ledger.
Enforces "agents propose, do not self-apply" at the DB boundary.
- Per-type handlers: revise_strategy (versioned strategy + node replace),
add_edge, attach_evidence (requires existing source_id; source minting is
intentionally out of scope for kb_apply's grants).
- Strict apply_payload contract (v1); freeform eval packets are normalized
upstream, not applied directly.
- --dry-run prints exact SQL; idempotent (refuses non-approved / already-applied);
transactional with an in-txn DO-block invariant check that rolls back on failure.
- Unit tests cover SQL builders, validation, dispatch, and status guards.
* fix(kb): rowcount=1 apply guard + real applied_by FK stamp
Closes the three draft-exit review items on the apply engine:
- Ledger flip now runs in a DO block asserting exactly one 'approved'
row moved to 'applied' (GET DIAGNOSTICS row_count). Closes the
concurrent double-apply race — load_proposal (read) and the flip
(write) are separate statements, so a row lock cannot span them; only
one concurrent apply can match status='approved', so rowcount=1 is the
authoritative guard. A loser RAISEs and the whole txn rolls back.
- applied_by_agent_id is stamped as a real FK resolved from public.agents
by handle, defaulting to the kb-apply service agent — no more NULL FK,
no backfill needed.
- scripts/kb_apply_prereqs.sql: one-time superuser bootstrap — inserts the
kb-apply service-agent row (kb_apply never gets INSERT on agents), grants
kb_apply SELECT on public.agents, and ensures the one-active-strategy
unique index (idempotent; already present on prod).
18/18 unit tests pass.
* fix(kb): hard-resolve applied_by handle, RAISE on NULL FK
Resolve applied_by into a variable and assert NOT NULL before the ledger
flip, instead of an inline subselect that silently stamps a NULL
applied_by_agent_id on an unresolved handle. Since the FK is ON DELETE SET
NULL, a bad handle (typo/unseeded agent) was a legal silent NULL -- the
perpetually-NULL FK we eliminated. Unresolved handle now hard-fails ->
rollback. Non-default --applied-by (operator, future drafters) is the path
that goes through the lookup and could strand NULL.
* Wire Leo Telegram x402 smart research
* Suppress token-bearing Telegram HTTP logs
* Keep Telegram typing visible during Leo proxy calls
* Allow Leo Telegram social research spend cap
* Route contextual Leo research prompts to smart research
* Generalize Leo smart research intent routing
* Resume Leo smart research from paid work orders
- Encode transcript requirements for model discovery and Pentagon boundary
- Add KB read/propose skill for Hermes, OpenClaw, and Claude-style agents
- Extend LLM contract checks; verify with 422-test suite
`.agents/skills/living-ip-kb-interop/SKILL.md`
`.agents/skills/nousresearch-hermes-agent/SKILL.md`
`.agents/skills/openclaw-agent/SKILL.md`
`docs/llm-refinement-decision-engine.md`
`scripts/check_llm_refinement_contract.py`
- Define Rio and Theseus as economics and model-integrity evaluators
- Add DB, Hermes, and OpenClaw skills with no-secret defaults
- Gate CI on LLM refinement contracts; verify with 422-test suite
`.agents/skills/decision-engine-refinement/SKILL.md`
`.agents/skills/nousresearch-hermes-agent/SKILL.md`
`.agents/skills/openclaw-agent/SKILL.md`
`.agents/skills/teleo-db-operator/SKILL.md`
`.crabbox.yaml`
`.github/workflows/ci.yml`
`docs/llm-refinement-decision-engine.md`
`scripts/check_llm_refinement_contract.py`
reweave.py and ingestion run as the operator Forgejo token, so the prior
opener-based classifier set submitted_by=m3taversal for every system
maintenance PR. backfill_submitted_by.py never overrides non-NULL rows,
so this misattribution accumulated: ~2,748 reweave/ingestion PRs and
~3,706 <agent>/ research/entity PRs were credited to the operator on
the leaderboard and contribution_events table.
Two parts:
1. lib/merge.py: at PR discovery, classify by branch prefix first.
reweave/, ingestion/ -> submitted_by = 'pipeline'
<agent>/ (per _AGENT_NAMES) -> submitted_by = '<agent>'
otherwise human -> submitted_by = author.lower()
otherwise pipeline -> submitted_by = None
(extract.py sets from proposed_by)
Origin flag updated so domain detection and priority still fire for
branch-classified pipeline PRs. Human PRs lowercased to maintain the
canonical-handle contract enforced in PR #9.
2. scripts/reattribute-by-branch-prefix.py: historical cleanup.
Per affected PR (atomic):
- UPDATE prs.submitted_by -> target
- UPDATE sources.submitted_by where source_path matches
- UPDATE contribution_events handle ('m3taversal',role='author')
-> target, kind='agent'. Collision (target already has author
event for PR) deletes the m3ta row; target wins.
Scope is deliberately conservative: extract/ branches stay attributed
to m3taversal because proposed_by-missing legitimately defaults to the
operator (telegram drops). Only reweave/, ingestion/, and <agent>/.
Dry-run shows 6,454 PRs + 284 events to move. Pre-flight collision
query returns 0; pre-flight kind check confirms m3ta has only role=author
events on this set (no challenger/synthesizer/evaluator).
Idempotent. Dry-run by default. Run with --apply after deploy + DB
snapshot.
Companion / write-side fix to fix/activity-feed-canonical-handle.
The activity-feed canonicalization was a read-side guard. The bug at the
source is that extract.py and two backfill scripts write decorated
strings (Vida (self-directed), pipeline (reweave), @m3taversal) into
prs.submitted_by and sources.submitted_by. Downstream readers
(lib.contributor.insert_contribution_event, scripts/scoring_digest,
diagnostics/activity_feed_api) all strip the decorator on read — but
anything that reads the column verbatim (like /api/activity-feed before
the read-side fix) 404s on /contributors/{decorated-handle}.
Stop writing the decorator. The self-directed signal is already carried
by intake_tier == research-task plus the prs.agent column; the suffix
is redundant string noise that costs us correctness at every consumer
that forgets to strip.
Changes:
- lib/extract.py:690 — write canonical handle via attribution.normalize_handle.
Direct elif for intake_tier == research-task now stores just agent_name.
@m3taversal -> m3taversal.
- diagnostics/backfill_submitted_by.py — same fix in two branches plus
the reweave branch (pipeline (reweave) -> pipeline).
- scripts/backfill-research-session-attribution.py — UPDATE prs sets
agent handle alone, no suffix. Docstring + log line updated.
- scripts/normalize-submitted-by.py (new) — one-time backfill that
canonicalizes existing prs.submitted_by and sources.submitted_by rows.
Strips trailing parenthetical decorators, lowercases, drops @. Defaults
to dry-run; --apply to commit. Skips rows that would normalize to
invalid handles (no garbage falls through silently).
Dry-run against live pipeline.db:
prs: 3008 rows need normalization (clean mappings, 0 invalid)
sources: 730 rows need normalization (clean mappings, 0 invalid)
Total: 3738 rows. All map to existing handle column values.
After this lands + auto-deploys, the operator should run
python3 scripts/normalize-submitted-by.py --apply
once to clean historical rows. The read-side canonicalization in
diagnostics/activity_feed_api.py (fix/activity-feed-canonical-handle)
becomes redundant defense-in-depth instead of load-bearing.
No KB writes.
3 nits from review of d60b6f8 + Q4 ask:
1. test_window_24h_only_today: replace always-true assertion with
concrete `assert handles == ["carol"]`. Push alice's most-recent
event from -1 days to -2 days to eliminate fixture-vs-query
microsecond drift on the 24h boundary.
2. _call helper: asyncio.get_event_loop().run_until_complete →
asyncio.run (deprecation in 3.12, raises in some 3.14 contexts).
3. test_invalid_limit_falls_to_default: dead first call removed,
misleading "7 entries" comment now matches assertion.
Q4: scripts/reset-m3taversal-sourcer.py captures the surgical
UPDATE we ran on VPS as a reviewable artifact. Idempotent (no-op
on already-reset rows), audit_log entry per run. Ganymede's point:
DB mutations should leave a code paper trail, not just an audit
row whose origin lives only in the executor's memory.
30/30 tests pass on VPS hermes venv (aiohttp 3.13.5, py 3.11.15).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two-part fix for a bug where every claim extracted from agent overnight
research sessions was being credited to m3taversal in contribution_events
(visible in the activity feed as "@m3taversal" on agent-derived claims).
Forward fix (research/research-session.sh):
The frontmatter template the agent prompt instructs Claude to use now
includes `proposed_by: ${AGENT}` and `intake_tier: research-task`. With
those fields present, extract.py path 1 (line 687) takes precedence and
sets prs.submitted_by to the agent handle, which then propagates into
contribution_events as a kind='agent' author event for the agent.
Without the fields, extract.py fell through to the default branch on
line 695 and set submitted_by='@m3taversal'.
Backfill (scripts/backfill-research-session-attribution.py):
Identifies research-session-derived PRs by finding teleo-codex commits
matching `^<agent>: research session YYYY-MM-DD —`, listing the
inbox/queue/*.md files added in each commit's diff, and matching those
filename basenames against prs.source_path. Only PRs currently
submitted_by='@m3taversal' AND merged within the configurable window
are touched. Default --dry-run; --apply to commit.
For each match the script:
1. UPDATE prs SET submitted_by = '<agent> (self-directed)'
2. INSERT OR IGNORE the agent author event (kind='agent', weight=0.30)
with the original PR's domain, channel, merged_at preserved
3. DELETE the misattributed m3taversal author event
Applied 30-day backfill on VPS:
- 304 PRs re-attributed (rio 74, clay 70, astra 53, vida 48,
theseus 30, leo 29)
- 297 m3taversal author events deleted, 304 agent author events
inserted (delta of 7 = pre-v24 PRs that never had m3ta events
in the first place; we still create the new agent event)
- m3taversal author count: 1368 → 1071 (−22%)
- Pre-backfill DB snapshot: pipeline.db.bak-pre-research-attribution
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1. WARNING — orphan contributor_aliases after publisher/garbage delete:
Added alias cleanup to the transaction (gated on --delete-events, same
audit rationale as events). Both garbage and publisher deletion loops
now DELETE matching contributor_aliases rows. Dry-run adds an orphan
count diagnostic so the --delete-events decision is informed.
2. NIT — inserted_publishers counter over-reports on replay:
INSERT OR IGNORE silently skips name collisions, but the counter
incremented unconditionally. Now uses cur.rowcount so a second apply
reports 0 inserts instead of falsely claiming 100. moved_to_publisher
set remains unconditional — publisher rows already present still need
the matching contributors row deleted.
3. NIT — handle-length gate diverged from writer path:
Widened from {0,19} (20 chars) to {0,38} (39 chars) to match GitHub's
handle limit and contributor.py::_HANDLE_RE. Prevents future long-handle
real contributors from falling through to review_needed and blocking
--apply. Current data has 0 review_needed either way.
Bonus (Q5): Added audit_log entry inside the transaction. One row in
audit_log.stage='schema_v26', event='classify_contributors' with counter
detail JSON on every --apply run. Cheap audit trail for the destructive op.
Verified end-to-end on VPS DB snapshot:
- First apply: 100/9/9/100/0 (matches pre-fix)
- Second apply: 0/9/0/0/0 (counter fix working)
- With injected aliases + --delete-events: 2 aliases deleted, 1 pre-existing
orphan correctly left alone (outside script scope), audit_log entry
written with accurate counters.
Ganymede msg-3. Protocol closed.
Separates three concerns currently conflated in contributors table:
contributors — people + agents we credit (kind in 'person','agent')
publishers — news orgs / academic venues / platforms (not credited)
sources — gains publisher_id + content_type + original_author columns
Rationale (Cory directive Apr 24): livingip.xyz leaderboard was showing CNBC,
SpaceNews, TechCrunch etc. at the top because the attribution pipeline credited
news org names as if they were contributors. The mechanism-level fix is a
schema split — orgs live in publishers, individuals in contributors, each
table has one semantics.
Migration v26:
- CREATE TABLE publishers (id PK, name UNIQUE, kind CHECK IN
news|academic|social_platform|podcast|self|internal|legal|government|
research_org|commercial|other, url_pattern, created_at)
- CREATE TABLE contributor_identities (contributor_handle, platform CHECK IN
x|telegram|github|email|web|internal, platform_handle, verified, created_at)
Composite PK on (platform, platform_handle) + index on contributor_handle.
Enables one contributor to unify X + TG + GitHub handles.
- ALTER TABLE sources ADD COLUMN publisher_id REFERENCES publishers(id)
- ALTER TABLE sources ADD COLUMN content_type
(article|paper|tweet|conversation|self_authored|webpage|podcast)
- ALTER TABLE sources ADD COLUMN original_author TEXT
(free-text fallback, e.g., "Kim et al." — not credit-bearing)
- ALTER TABLE sources ADD COLUMN original_author_handle REFERENCES contributors(handle)
(set only when the author is in our contributor network)
- ALTER wrapped in try/except on "duplicate column" for replay safety
- Both SCHEMA_SQL (fresh installs) + migration block (upgrades) updated
- SCHEMA_VERSION bumped 25 -> 26
Migration is non-breaking. No data moves yet. Existing publishers-polluting-
contributors row state is preserved until the classifier runs. Writer routing
to these tables lands in a separate branch (Phase B writer changes).
Classifier (scripts/classify-contributors.py):
Analyzes existing contributors rows, buckets into:
keep_agent — 9 Pentagon agents
keep_person — 21 real humans + reachable pseudonymous X/TG handles
publisher — 100 news orgs, academic venues, formal-citation names,
brand/platform names
garbage — 9 parse artifacts (containing /, parens, 3+ hyphens)
review_needed — 0 (fully covered by current allowlists)
Hand-curated allowlists for news/academic/social/internal publisher kinds.
Garbage detection via regex on special chars and length > 50.
Named pseudonyms without @ prefix (karpathy, simonw, swyx, metaproph3t,
sjdedic, ceterispar1bus, etc.) classified as keep_person — they're real
X/TG contributors missing an @ prefix because extraction frontmatter
didn't normalize. Cory's auto-create rule catches these on first reference.
Formal-citation names (Firstname-Lastname form — Clayton Christensen, Hayek,
Ostrom, Friston, Bostrom, Bak, etc.) classified as academic publishers —
these are cited, not reachable via @ handle. Get promoted to contributors
if/when they sign up with an @ handle.
Apply path is transactional (BEGIN / COMMIT / ROLLBACK on error). Publisher
insert happens before contributor delete, and contributor delete is gated
on successful insert so we never lose a row by moving it to a failed
publisher insert.
--apply path flags:
--delete-events : also DELETE contribution_events rows for moved handles
(default: keep events for audit trail)
--show <handle> : inspect a single row's classification
Smoke-tested end-to-end via local copy of VPS DB:
Before: 139 contributors total (polluted with orgs)
After: 30 contributors (9 agent + 21 person), 100 publishers, 9 deleted
contribution_events: 3,705 preserved
contributors <-> publishers overlap: 0
Named contributors verified present after --apply:
alexastrum (claims=6) thesensatore (5) cameron-s1 (1) m3taversal (1011)
Pentagon agent 'pipeline' (claims_merged=771) intentionally retained — it's
the process name from old extract.py fallback path, not a real contributor.
Classified as agent (kind='agent') so doesn't appear in person leaderboard.
Deploy sequence after Ganymede review:
1. Branch ff-merge to main
2. scp lib/db.py + scripts/classify-contributors.py to VPS
3. Pipeline already at v26 (migration ran during earlier v26 restart)
4. Run dry-run: python3 ops/classify-contributors.py
5. Apply: python3 ops/classify-contributors.py --apply
6. Verify: livingip.xyz leaderboard stops showing CNBC/SpaceNews
7. Argus /api/contributors unaffected (reads contributors directly, now clean)
Follow-up branch (not in this commit):
- Writer routing in lib/contributor.py + extract.py:
org handles -> publishers table + sources.publisher_id
person handles with @ prefix -> auto-create contributor, tier='cited'
formal-citation names -> sources.original_author (free text)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Addresses two findings in commit 762fd42 review:
1. BUG: guard query was tautological. `SELECT MAX(number) FROM prs WHERE
number < 900000` filters out exactly what the `>= 900000` check tests.
Replaced with a direct check for unexpected rows in the synthetic range
(excluding our known 900068/900088).
2. WARNING: origin defaults to 'pipeline' via schema default. lib/merge.py
convention is origin='human' for external contributors. Synthetic rows
now set origin='human', priority='high' — matches discover_external_prs
for real GitHub PRs. Prevents Phase B origin-based filtering from
misclassifying Alex/Cameron as machine-authored.
Also flagged in review: credit projection was optimistic. Author events are
PR-level (not per-claim), so Alex gets 1×0.30 author credit, not 6. Same
for Cameron. Per-claim originator credit goes to the 7 frontmatter sourcers
where applicable. Not a code change — expectation reset for Cory.
