2.7 KiB
2.7 KiB
Working Leo Blind Review Before Broad Hardening
Date: 2026-07-13
Status: not reliable for unattended operator use
Method
A 12-question temporary-profile handler run used broad prompts without known proposal IDs. It posted nothing to Telegram and made no database or service change. Two independent strict reviews evaluated the same transcript against the live v1 schema and retained proof.
Reviewer results differed in severity but agreed on the verdict:
- strict binary review:
1/12 pass; - pass/partial/fail review:
2 pass,4 partial,6 fail.
Converged Failures
- Replies presented target-architecture fields as current schema, including source author/channel/date, claim bodies, evidence excerpts, edge rationale, and forecast-resolution fields.
- Replies invented edge types such as
superseded_by,relates_to, andresolves. Current v1 usessupersedes;superseded_byis a claim column. - Replies treated unchanged canonical counts as a complete explanation of behavior, ignoring deployed skills, runtime configuration, rendered files, and persisted Hermes session state.
- A handler-only no-post run was described as proof of the live Telegram path.
- A temporary conversation label was promoted into source provenance rather than resolving a real URL, storage path, file hash, or retained artifact.
- Staging was treated as though it required the same authorization as canonical apply. Requested source capture and proposal staging are allowed; guarded canonical apply is the approval boundary.
- Forecast handling proposed rewriting historical confidence and inventing resolution storage that current v1 does not have.
- Claims/evidence were not consistently treated as shared knowledge while beliefs and positions remain agent-specific.
- Replies were too long for operational use:
9,033words total,753words per answer on average, with several over1,000words.
Repair Incorporated
- Explicit current-schema and edge contracts in both VPS and GCP bridge skills.
- Runtime/session/canonical proof-tier separation.
- Source-link versus provenance-quality distinction.
- Shared claims/evidence versus agent-specific positions.
- Forecast-history preservation and current schema gap.
- Autonomous source capture and staging boundary.
- Maximum response length in the out-of-sample scorer.
- Expansion from 9 to 15 broad prompts covering these failure modes.
Claim Ceiling
This is a pre-fix handler baseline. It is not Telegram-visible proof. The repair is not accepted until the deployed 15-question run passes, independent strict review converges, restart survival passes, and a separate Telegram-visible no-mutation canary succeeds.