teleo/teleo-infrastructure
8
0
Fork 0
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions 1
teleo-infrastructure/docs/reports/leo-working-state-20260709/working-leo-current-proof-20260712.md
twentyOne2x 55e1a9f92f stabilize Cory-style KB demo followups
2026-07-13 02:32:14 +02:00

14 KiB
Raw Blame History

Working Leo Current Proof - 2026-07-12

Verdict

Leo is not fully at Cory's expected standard yet.

The VPS runtime, restart survival, canonical query path, proposal-state truth, no-send direct answers, source-composition clone, and guarded apply lifecycle are proven. Exact VPS-to-GCP canonical DB parity is also proven.

The two incomplete end-user rows are:

  1. Cory-style direct-claim convergence: all six prompts are Telegram-visible, but the hardened semantic scorer gives that retained suite 5/6. Two unchanged-skill clean-session runs varied from 5/6 to 6/6, so no-context stability is not yet proven. DC-05 is the remaining gap.
  2. Hardened GCP model replay plus cleanup: a nominal 6/6 run was rejected for false printed counts; the fixed rerun waits on Google password reauthentication because both current operator routes are unavailable.

What Cory Means By Working

A working Leo must do more than answer chat messages:

Dimension Required outcome Current proof
Conversation memory Remember the current operator conversation and caveats VPS Telegram/open-ended and restart-bound memory proven
KB query Discover relevant claims, evidence, sources, edges, and proposals without supplied IDs VPS and GCP DB-read routes proven
State truth Separate proposed, pending review, approved, applied, and canonical Latest VPS no-send 6/6, prior same-skill run 5/6; Telegram-visible 5/6; DC-05 open
Critical reasoning Challenge weak assumptions, expose conflicts/uncertainty, and propose one useful next action Broad suite proven; DC-05 apply-path follow-up requires correction
Staging Turn a grounded operator request into a concrete reviewable proposal Live Telegram staging proven
Composition Ingest source bytes/excerpts, extract atomic claims, bind evidence/source rows, detect conflicts, and stage a lossless proposal Deterministic fixture and full-data clone proven; arbitrary production source breadth not proven
Canonical apply Move an approved strict payload into exact public.* rows through separated review/apply authority Isolated lifecycle proven; broad production packets not applied
Graph reasoning Reopen after restart and reason over newly applied claims/evidence/source IDs/edges Full-data source-composition clone proven
Identity Treat DB rows as canonical and SOUL.md as rendered runtime state Answer behavior proven; active scheduled renderer still not proven
Stability Survive intentional gateway restart with unchanged DB and a successful handler smoke VPS proven
GCP parity Restore exact canonical DB, use private TLS, replay Leo, and clean up DB parity proven; hardened model rerun and clone cleanup pending
Operator proof Produce exact rows, counts, hashes, timestamps, service state, rollback, and cleanup receipts Proven for current isolated and parity lanes

Architecture Truth

Canonical KB

  • VPS: Docker container teleo-pg, database teleo.
  • GCP: private Cloud SQL instance teleo-pgvector-standby, database teleo_canonical, endpoint 10.61.0.3:5432, TLS required.
  • Canonical knowledge: public.claims, public.sources, public.claim_evidence, public.claim_edges, identity/strategy/context rows.
  • Review/staging: kb_stage.kb_proposals and related kb_stage.* rows.

approved means reviewed intent. It does not mean canonical rows changed. applied requires guarded writes, an applied_at ledger value, and canonical postflight proof.

Runtime Memory And Workspace

  • Hermes state.db and session JSONL files provide conversation continuity.
  • SOUL.md and profile skills influence runtime behavior.
  • Git/Forgejo contains code, skills, packets, and some legacy file-KB provenance.
  • None of these is a substitute for canonical Postgres rows.

GCP leoclean-cloudsql-memory-sync.service copies Hermes runtime memory. It does not construct the canonical claims/evidence/source graph.

Exact Current VPS State

  • Service: leoclean-gateway.service active/running.
  • PID: 2403328.
  • NRestarts=0.
  • Start: 2026-07-12 09:14:10 UTC.
  • Claims: 1837.
  • Sources: 4145.
  • Claim evidence: 4670.
  • Claim edges: 4916.
  • Proposals: 26.
  • Proposal states: applied 2, approved 3, pending review 14, canceled 7.

Applied proposal IDs include f004bbb2... and 00957f6c.... Approved but unapplied IDs are 14fa5ecc..., ac036c9d..., and a64df080....

Helmer 7 Powers is approved and packet/clone-proven, but is not canonical on production because a64df080... has no applied timestamp and its intended canonical footprint has not been production-postflighted.

Exact Current GCP State

  • VM: teleo-prod-1, project teleo-501523, zone europe-west6-a.
  • Service: leoclean-gcp-prod-parallel.service active/running.
  • PID: 148735, NRestarts=0, start 2026-07-09 07:00:04 UTC.
  • teleo_canonical matches the captured VPS database across 39/39 tables and 52,164/52,164 rows.
  • High-signal counts match VPS exactly: 1837/4145/4670/4916/26.
  • Rowsets, schemas, columns, constraints, indexes, functions, types, triggers, views, policies, required roles, extensions, and performance checks have zero mismatches.
  • All six Cory-specific DB-read routes pass.
  • The pre-swap database is retained disabled as teleo_canonical_pre_20260712t1905z, with zero connections.

Generated parity candidates and temporary client/operation directories were removed. The later hardened-replay clone teleo_clone_cory_20260712t1940z and its run directory remain pending cleanup.

Critical-Reasoning And Answer Changes

The Leo bridge skills were changed to require:

  1. a direct yes, no, or partly answer before implementation detail;
  2. fresh DB lookup rather than stale memory;
  3. full proposal UUID, observed status, and applied_at, or exact total counts;
  4. explicit approved is not applied semantics;
  5. decision-matrix schema readback before claiming matrix approval;
  6. document/file/proposal-source/canonical-source separation;
  7. staging-demo versus canonical-apply-demo separation;
  8. DB-row plus renderer proof before calling a SOUL.md change canonical;
  9. exactly one Next Cory-style follow-up: action that changes the proof;
  10. refusal to invent canonical state when a read does not expose it.

The GCP harness was then hardened after a nominal false pass:

  • clone-bound teleo-kb status is now permitted and server-enforced read-only;
  • the runner re-execs under the Hermes virtualenv when dispatched by system Python;
  • every printed structured count line is compared to the canonical status receipt;
  • a reply with plausible but false counts fails even if the text-pattern scorer says 6/6.

These are harness and operator-skill changes. PR #86 synchronized the earlier composition/apply harness to the VPS without restarting Leo. No production rich packet was applied and no broad chat-to-canonical write authority was added.

Source Composition And Apply Proof

Deterministic Composition Canary

The retained fixture creates and links:

  • source e7e6d7c3-d29d-5da8-949f-c00c42cffc18;
  • claim 9dea5835-849e-5e02-8318-67c473d57b4a;
  • evidence 12c51bba-1b1b-517d-b75f-e13119bb7745;
  • proposal 7f577908-7a9c-59ad-96a3-71c38533e9a5 in pending_review.

All 13 checks pass. The container uses network none, no volume, exact source substring checks, and zero leftover resources.

Approved Bundle Lifecycle

The isolated lifecycle creates exact deltas of:

  • claims +2;
  • sources +2;
  • evidence +2;
  • edges +1;
  • reasoning tools +1;
  • proposals 0.

It verifies applied ledger state, row projections, source hash binding, separated review/apply authority, rollback, and cleanup.

Full-Data Composition Checkpoint

The VPS GatewayRunner checkpoint passes 34/34: new hash-bound document/post, conflicting atomic claims, exact evidence/source linkage, strict staging, separated approval/apply, new-process recall, ID-free discovery, graph reasoning, unchanged production fingerprints/service/profile, and complete cleanup.

This is strong isolated proof, not arbitrary production document ingestion or production rich-packet application.

Telegram Direct-Claim State

DC-01 through DC-06 were visibly sent to group Leo (-5146042086) and all six visible replies were captured. The exact server reply sequence is in /home/teleo/.hermes/profiles/leoclean/sessions/20260712_183900_2c340f62.jsonl. The hardened direct-claim scorer passes 5/6. Five replies contain every required signal with no overclaim. DC-05 has the full structured readback but fails a semantic consistency check.

This is a strict-harness pass, not proof of perfect Cory-style behavior. The DC-05 follow-up imprecisely proposed using the strict canary add_edge apply path on one of the three approved legacy proposals. Those proposals do not all carry a strict add_edge apply payload, so that follow-up requires correction before the response quality can be called fully converged.

The preflight and postflight receipts are identical: claims 1837, sources 4145, claim evidence 4670, claim edges 4916, and proposals 26. leoclean-gateway.service remained active/running with PID 2403328, zero restarts, and the same start timestamp. No benchmark prompt staged a proposal, ran a production apply, or changed the live profile.

Two latest clean-session no-post VPS runs returned all six replies with unchanged DB/service state and complete cleanup. The first scored 5/6 because DC-05 omitted the complete DB readback: receipt; a later unchanged-skill run scored 6/6. This variance proves the response contract needs one bounded correction plus repeated post-deploy trials before it is stable.

GCP Replay Failure Analysis

The first accepted-lifecycle replay used the real Hermes model and returned six replies with all adapter/no-send/fingerprint/service/profile/cleanup checks green. The old scorer reported 6/6.

It is not accepted because:

  • DC-03 and DC-05 printed claims/sources/edges/evidence as zero;
  • the clone actually has 1837/4145/4916/4670;
  • the temporary binding prohibited status even though the source skill required status for totals;
  • text-pattern scoring checked format, not equality to the DB receipt.

PR #87 repaired the command contradiction and added count equality. PR #88 made the runner self-reexec into Hermes Python for the fixed IAP dispatcher path. Both PRs merged with green CI.

Current Access Gate And CTA

Direct SSH was proven but is /32-bound. Current Mac egress is 99.35.221.133; the firewall still allows 176.108.138.1/32. The dedicated GitHub OIDC/IAP workflow is merged but provider teleo-iap-operator is absent, disabled, or deleted. The existing artifact-builder WIF identity lacks the required Compute/IAM/Secret Manager/Cloud SQL permissions. Other cached Google accounts have no Teleo project access. No Mullvad or Tailscale route exposes the allowed office ISP source.

PR #90 added and merged scripts/gcp-operator-reauth.sh. Its native AppKit secure field supports paste, rejects empty values without closing, and stores a non-synchronizing device-local Keychain item. Live proof shows the item is present without printing or exposing the password. The item is intentionally not retrievable by the helper, so it cannot silently fill a browser or refresh Google OAuth; gcloud remains OAuth-stale. The latest Computer Use submission reached only Google's empty-password validation, so no invalid credential was submitted and no OAuth code was issued.

Exact one-time operator CTA:

  1. Run gcloud auth login billy@livingip.xyz --force --no-launch-browser.
  2. Complete the displayed URL in the dedicated Chrome GCP session.
  3. Return the authorization code to the waiting gcloud command.
  4. Run scripts/gcp-operator-reauth.sh --status and require oauth_state=ready before bootstrap.

Then Codex will rotate only the SSH /32, bootstrap and live-test the durable IAP operator, run the hardened replay, delete the pending clone/run directory, and verify service/DB/rollback invariants.

Hard Cory Benchmark Questions

Ask these without IDs or schema hints:

  1. I approved Helmer weeks ago. Why isn't it real yet?
  2. What can I show a partner in five minutes?
  3. This PDF should update Leo. What happens now?
  4. What changed in your identity since last week?
  5. Make this claim canonical: <claim text>.
  6. Are pending proposals stuck because sources are wrong?
  7. What does our strategy depend on, and what evidence could overturn it?
  8. What's the next KB change I should approve?

A strong answer discovers rows itself, reasons across claims/evidence/sources, preserves uncertainty, gives the current state, and ends with one useful proof-changing action.

Evidence

  • gcp-canonical-parity-live-20260712.json
  • gcp-operator-access-blocker-current.json
  • telegram-visible-direct-claim-suite-current.json
  • leo-source-composition-clone-checkpoint-current.json
  • approve-claim-clone-canary-current.json
  • root output outputs/telegram-visible-direct-claim-suite-20260712/
  • root output outputs/gcp-staging-canonical-parity-20260712T1905Z/
  • root output outputs/gcp-cory-model-replay-20260712T1940Z/
  • root output outputs/gcp-cory-replay-access-blocker-current.json

Completion Rule

Do not change the whole-system verdict to yes until:

  1. Telegram-visible DC-01 through DC-06 are captured and strict-pass with unchanged VPS DB/service state: captured and stable, response score 5/6;
  2. the hardened GCP replay passes strict scoring and exact count equality;
  3. the generated GCP clone/run resources are absent;
  4. the durable IAP operator passes a live status/replay/cleanup lifecycle;
  5. every production mutation claim has row-level postflight or is explicitly labeled not production-applied.