Canary

Run one isolated stateless challenger against one persistent temporary-profile Leo GatewayRunner session. The challenger asks for an evidence-bounded claim, attacks its weakest causal step, and forces a narrower review-only revision. The run must leave Telegram, the canonical database, the deployed profile, and the gateway unchanged.

What changed

Added a six-turn C1/L1/C2/L2/C3/L3 live protocol with exact transcript-prefix binding.
Added a stateless Gemini challenger with no profile, memory, tools, or database access.
Added a temporary Leo profile with model tools disabled, a fail-closed read-only KB wrapper, and a canary-only context policy that cannot replace the final objection response with an unrelated compiled capability response.
Added per-turn model/session/process/profile bindings, database retrieval receipts, and a review-only proposal packet.
Added five executed negative controls: collusion/leaked ID/silent apply, repeated execution identity, unbound row reference, forbidden or unknown mutation tool, and unrelated keyword-stuffed revision.
Added transport/source attribution, recovery readback, full database fingerprint comparison, service/behavior invariance, cleanup verification, focused tests, and retained live evidence.

Operator command

python3 -B scripts/run_leo_agent_challenger_loop.py

The lane used the preserved project virtual environment because system Python does not include pytest.

Evidence

Live run: 7c6a2937042c
Runtime source commit: af5af06130c493dd4c99c9e53638fc5b239dbee3
Source report SHA-256: 343f18688334d5166bbb951f8672ccf4ff5a3a2b14e05f5279d3e1165b76f83b
Recovered raw remote report SHA-256: 6a7a9160095e77e5c42b7d8a2c995d3eb1e84f40624033a94ad49d616bb548a8
Proposal packet SHA-256: 89c543b8a12e922044f93bcc6b1e23a3a252b044dc1d45163575b70bc1a3c100
Database fingerprint SHA-256: 32a1f2c18f4b4c623443cbd4d9520982e9f291ec74ca63855cef51c88ac56f24
Standalone verifier: pass, current tier T2_runtime, zero failed checks.
Negative controls: 5/5 caught.
Independent post-live judge: ACCEPT — T2_runtime, no blocking findings.
Focused suite: 36 passed.
Merged origin/main commit 15c30f1fbe30c8f2295ddc33e293a45788a95706 without rewriting the feature commits.
Post-merge focused suite: 36 passed in 0.19 seconds.
Post-merge full suite: 1297 passed in 106.35 seconds.
Ruff and git diff --check: pass.

Retained files:

docs/reports/leo-working-state-20260709/leo-agent-challenger-loop-source-current.json
docs/reports/leo-working-state-20260709/leo-agent-challenger-loop-receipt-current.json
docs/reports/leo-working-state-20260709/leo-agent-challenger-loop-negative-controls-current.json
docs/reports/leo-working-state-20260709/leo-agent-challenger-loop-ledger.jsonl

Cleanup and no-write readback

Full 39-table canonical database fingerprint unchanged before/after.
Gateway PID, start timestamp, active/sub states, and NRestarts=0 unchanged.
Deployed behavior manifest unchanged for the accepted run.
No model tool calls; mutation preflight rejected with exit 77.
No Telegram post and no canonical or staging mutation by the harness.
Proposal remains pending_review; review incomplete; apply unauthorized and unexecuted.
Leo worker stopped gracefully with exit 0; no orphan worker; temporary roots removed.
Initial report fetch timed out after the completed run. The exact remote file was recovered by SCP, hashed, then the remote path was unlinked and verified absent. The recovery contract is retained in the source report.

Cost and authorization boundary

Successful challenger usage: 2,201 prompt + 361 completion tokens; configured Gemini Flash estimate about $0.00055.
Successful Leo usage: three OpenRouter Sonnet calls, 46,198 total reported tokens. At the repo's uncached list rates, this is an upper-bound estimate of about $0.17; provider cache accounting may reduce the actual charge.
The command authorizes only this bounded no-send/no-write canary. It does not authorize Telegram delivery, proposal staging, approval, apply, production deployment, or canonical mutation.

Not shipped

T3 deployment of this orchestrator.
Human acceptance of the revised proposal.
Telegram-visible delivery.
Proposal staging, approval, or canonical apply.

Review focus

Verify the temporary-profile isolation and context-only patch cannot affect the live profile.
Verify semantic linkage rejects unrelated revisions without overfitting exact prose.
Verify transport recovery/source attribution and the five negative controls.
Verify the claim remains capped at one T2 no-send/no-write exchange.

Delivery and worktree lifecycle

Delivery state: open PR #152, mergeable at creation.
Worktree state: retained_unintegrated while PR #152 is open.
Per WORKTREE_LIFECYCLE.md, this worker does not remove or prune the worktree. The integration owner owns cleanup only after merge, deliberate port, or explicit rejection is verified.

5 KiB Raw Blame History