Canary

Run one isolated stateless challenger against one persistent temporary-profile Leo GatewayRunner session. The challenger asks for an evidence-bounded claim, attacks its weakest causal step, and forces a narrower review-only revision. The run must leave Telegram, the canonical database, the deployed profile, and the gateway unchanged.

Changes-required repair checkpoint

Current tier is partial_lower_tier until one fresh clean, pushed, current-main T2 run passes the repaired gate.
C3 now uses one shared runner/verifier predicate. Measurement-only objections never advance to Leo; the runner regenerates up to three times and retains every rejected model-call trace without adding it to the six-turn transcript.
The verifier validates the raw submitted list before ID mapping: exactly six object rows, exact ordered unique IDs, and whole-report execution, tool, mutation/write, database, and row-reference provenance.
The exact injected seventh X1 positive-mutation turn is the sixth executable negative control and is mirrored in fixtures/working-leo/agent-challenger-loop-negative-controls-v1.json.
Failed current-main run 46b8b7834e35 is preserved under immutable run-specific names with source SHA-256 81d71f5db918f4be0e0308fa060fa244480739ac00cbfd1bef0f6fce07421d0a.
Current-main repair checks pass after merging origin/main at 33399bd9acfa09e4e5409873e86c8c46ac694a8d: 52 expanded relevant and 1,536 full-suite tests under the isolated dev environment. All four reviewer-named Python files pass Ruff lint and format checks. Fresh current-main runtime evidence remains pending.

What changed

Added a six-turn C1/L1/C2/L2/C3/L3 live protocol with exact transcript-prefix binding.
Added a stateless Gemini challenger with no profile, memory, tools, or database access.
Added a temporary Leo profile with model tools disabled, a fail-closed read-only KB wrapper, and a canary-only context policy that cannot replace the final objection response with an unrelated compiled capability response.
Added per-turn model/session/process/profile bindings, database retrieval receipts, and a review-only proposal packet.
Added six executed negative controls: collusion/leaked ID/silent apply, repeated execution identity, unbound row reference, forbidden or unknown mutation tool, an unexpected seventh positive-mutation turn, and unrelated keyword-stuffed revision.
Added transport/source attribution, recovery readback, full database fingerprint comparison, service/behavior invariance, cleanup verification, focused tests, and retained live evidence.

Operator command

python3 -B scripts/run_leo_agent_challenger_loop.py

The lane used the preserved project virtual environment because system Python does not include pytest.

Historical pre-review evidence

The following accepted run predates the changes-required predicates and is retained as history, not current closure.

Live run: 7c6a2937042c
Runtime source commit: af5af06130c493dd4c99c9e53638fc5b239dbee3
Retention-safe source report SHA-256: 4fd4269b795c32629d4b250cb66d56ceadaa93eaaa1f8f00a5ec19d76bf4f09e
Recovered raw remote report SHA-256: 6a7a9160095e77e5c42b7d8a2c995d3eb1e84f40624033a94ad49d616bb548a8
Proposal packet SHA-256: 89c543b8a12e922044f93bcc6b1e23a3a252b044dc1d45163575b70bc1a3c100
Database fingerprint SHA-256: 32a1f2c18f4b4c623443cbd4d9520982e9f291ec74ca63855cef51c88ac56f24
Standalone verifier: pass, current tier T2_runtime, zero failed checks.
Historical negative controls: 5/5 caught under the pre-review verifier.
Independent post-live judge: ACCEPT — T2_runtime, no blocking findings.
Focused suite: 36 passed.
Merged origin/main commit 15c30f1fbe30c8f2295ddc33e293a45788a95706 without rewriting the feature commits.
Post-merge focused suite: 36 passed in 0.19 seconds.
Post-merge full suite: 1297 passed in 106.35 seconds.
Ruff and git diff --check: pass.
Post-hygiene focused suite: 39 passed in 0.25 seconds.

Retained files:

docs/reports/leo-working-state-20260709/leo-agent-challenger-loop-source-current.json
docs/reports/leo-working-state-20260709/leo-agent-challenger-loop-receipt-current.json
docs/reports/leo-working-state-20260709/leo-agent-challenger-loop-negative-controls-current.json
docs/reports/leo-working-state-20260709/leo-agent-challenger-loop-ledger.jsonl

Cleanup and no-write readback

Full 39-table canonical database fingerprint unchanged before/after.
Gateway PID, start timestamp, active/sub states, and NRestarts=0 unchanged.
Deployed behavior manifest unchanged for the accepted run.
No model tool calls; mutation preflight rejected with exit 77.
No Telegram post and no canonical or staging mutation by the harness.
Proposal remains pending_review; review incomplete; apply unauthorized and unexecuted.
Leo worker stopped gracefully with exit 0; no orphan worker; temporary roots removed.
Initial report fetch timed out after the completed run. The exact remote file was recovered by SCP, hashed, then the remote path was unlinked and verified absent. The recovery contract is retained without the private path value.
The retained source projects both behavior manifests to their aggregate hashes, counts, and byte totals. It removes 2,658 per-file inventory rows and all absolute runtime paths while retaining the raw remote report hash and verifier contracts. The committed source is 1,402 lines rather than the original 17,330-line transport report.

Cost and authorization boundary

Successful challenger usage: 2,201 prompt + 361 completion tokens; configured Gemini Flash estimate about $0.00055.
Successful Leo usage: three OpenRouter Sonnet calls, 46,198 total reported tokens. At the repo's uncached list rates, this is an upper-bound estimate of about $0.17; provider cache accounting may reduce the actual charge.
The command authorizes only this bounded no-send/no-write canary. It does not authorize Telegram delivery, proposal staging, approval, apply, production deployment, or canonical mutation.

Not shipped

T3 deployment of this orchestrator.
Human acceptance of the revised proposal.
Telegram-visible delivery.
Proposal staging, approval, or canonical apply.

Review focus

Verify the temporary-profile isolation and context-only patch cannot affect the live profile.
Verify semantic linkage rejects unrelated revisions without overfitting exact prose.
Verify transport recovery/source attribution and the six negative controls.
Verify the claim remains capped at one T2 no-send/no-write exchange.

Integration hygiene

Removed the lane-local root goal.md; the durable control copy remains outside the repository.
Regenerated the retained source, verifier receipt, negative-control receipt, and ledger entry from the accepted live run rather than discarding the existing receipt changes.
Reduced the retained source from 17,330 to 1,402 lines by removing 2,658 duplicated per-file behavior inventory entries. Aggregate behavior/component hashes, counts, byte totals, raw remote report hash, and role contracts remain.
Targeted scan of the retained source: zero high-confidence secrets, absolute runtime paths, emails, or IPv4 values.
Targeted scan of the complete PR delta: zero high-confidence secrets, local private paths, emails, or IPv4 values. The 28 remaining path matches are required remote runtime constants or synthetic fail-closed test fixtures.
The historical standalone verifier passed at T2_runtime with all five pre-review negative controls caught; a fresh six-control current-main receipt is required before closure.

Delivery and worktree lifecycle

Delivery state: open PR #152, mergeable at creation.
Worktree state: retained_unintegrated while PR #152 is open.
Per WORKTREE_LIFECYCLE.md, this worker does not remove or prune the worktree. The integration owner owns cleanup only after merge, deliberate port, or explicit rejection is verified.

7.9 KiB Raw Blame History