Merge pull request #145 from living-ip/codex/gcp-cloudsql-fail-closed-postrestart-20260714

Keep GCP knowledge reads on Cloud SQL after restart

.agents/skills/teleo-gcp-parity-ops/SKILL.md

@@ -71,7 +71,15 @@ Do not describe a passing Hermes memory sync as canonical KB parity.
   retrieval receipts.
 - The generated clone was deleted. The retained rollback database remains
   connection-disabled with zero sessions. The GCP gateway remained PID
-  `148735`, `NRestarts=0`, active/running throughout.
+  `148735`, `NRestarts=0`, active/running throughout that bounded experiment.
+- PR `#144` merged the reviewed helper/skill. After deployment, the service
+  survived a controlled restart and is active/running at PID `304036`, start
+  time `2026-07-14 10:40:17 UTC`. Live post-restart `status` and `search`
+  returned Cloud SQL retrieval receipts with unchanged canonical counts.
+- A live regression showed the old wrapper could time out its stronger status
+  probe and fall through to a different local tool. Supported GCP KB commands
+  now route directly to Cloud SQL and fail closed on errors; two behavioral
+  tests enforce that invariant.
 - Persistent GCP `teleo_canonical` remains the older staging copy measured at
   `52,164` rows and `26` proposals. It has not been promoted or cut over.
 
@@ -82,6 +90,7 @@ Primary retained proof:
 - `docs/reports/leo-working-state-20260709/gcp-db-first-parity-current.json`
 - `docs/reports/leo-working-state-20260709/gcp-db-first-blind-claim-current.json`
 - `docs/reports/leo-working-state-20260709/gcp-db-first-cleanup-current.json`
+- `docs/reports/leo-working-state-20260709/gcp-db-first-live-deploy-restart-current.json`
 
 ## Required Parity Rows
 

.agents/skills/teleo-leo-onboarding/SKILL.md

@@ -27,9 +27,11 @@ Orient the worker before action. Build a current, proof-linked understanding of
 - GCP is a separate lane. Direct passwordless SSH and `sudo -n` work as of
   2026-07-14. The newest VPS DB was restored to a disposable private Cloud SQL
   clone with exact parity, and a real ID-free no-send reasoning turn passed.
-  Persistent GCP `teleo_canonical` is still an older staging copy; Telegram
+  The merged helper/skill were deployed, supported KB commands now fail closed
-  delivery, GCP canonical mutation, ongoing replication, promotion, and cutover
+  to Cloud SQL, and the service survived a controlled restart with live
-  remain separate proof rows.
+  post-restart status/search receipts. Persistent GCP `teleo_canonical` is still
+  an older staging copy; Telegram delivery, GCP canonical mutation, ongoing
+  replication, promotion, and cutover remain separate proof rows.
 
 ## Architecture Boundaries
 
@@ -53,11 +55,12 @@ From the repo root, read:
 3. `docs/reports/leo-working-state-20260709/gcp-db-first-parity-current.json`
 4. `docs/reports/leo-working-state-20260709/gcp-db-first-blind-claim-current.json`
 5. `docs/reports/leo-working-state-20260709/gcp-db-first-cleanup-current.json`
-6. `docs/reports/leo-working-state-20260709/operator-surface-map.md`
+6. `docs/reports/leo-working-state-20260709/gcp-db-first-live-deploy-restart-current.json`
-7. `docs/reports/leo-working-state-20260709/working-leo-definition-20260709.md`
+7. `docs/reports/leo-working-state-20260709/operator-surface-map.md`
-8. `.agents/skills/working-leo-m3taversal-outcomes/SKILL.md`
+8. `docs/reports/leo-working-state-20260709/working-leo-definition-20260709.md`
-9. `docs/reports/leo-working-state-20260709/approve-claim-clone-canary-current.md`
+9. `.agents/skills/working-leo-m3taversal-outcomes/SKILL.md`
-10. `docs/reports/leo-working-state-20260709/gcp-cloud-sql-t3-live-readonly-current.md`
+10. `docs/reports/leo-working-state-20260709/approve-claim-clone-canary-current.md`
+11. `docs/reports/leo-working-state-20260709/gcp-cloud-sql-t3-live-readonly-current.md`
 
 ## Required Status Split
 

.agents/skills/working-leo-m3taversal-outcomes/SKILL.md

@@ -133,6 +133,10 @@ is green at its required tier:
 - The GCP helper now gives deterministic receipts for every read, handles a
   canonical-only restored DB without the optional audit schema, and returns an
   honest zero-hit result instead of crashing.
+- The reviewed helper/skill are merged and deployed. GCP survived a controlled
+  restart, then live `status` and `search` reached persistent Cloud SQL with
+  unchanged counts and stable receipts. Supported commands fail closed instead
+  of falling back to a different local database.
 - The disposable GCP database and temporary profiles/processes were removed.
   Persistent GCP `teleo_canonical` remains the older staging copy and has not
   been promoted or cut over.

docs/reports/leo-working-state-20260709/current-truth-index.md

@@ -15,7 +15,17 @@ section below.
   claim plus both expected source rows, passed `18/18` runtime checks and `6/6`
   reasoning outcomes, made no DB write, and sent no Telegram message.
 - The disposable clone was deleted. The rollback DB remains disabled with zero
-  connections. The live GCP gateway stayed PID `148735`, `NRestarts=0`.
+  connections. The live GCP gateway stayed PID `148735`, `NRestarts=0` during
+  the disposable restore/reasoning/cleanup run.
+- PR `#144` then merged the database-first helper and skill. They were deployed
+  to GCP and the service survived a controlled restart: active/running at PID
+  `304036`, start time `2026-07-14 10:40:17 UTC`.
+- Post-restart status and search both routed to persistent Cloud SQL and emitted
+  retrieval receipts. The search found the expected claim and both expected
+  source IDs/hashes. Canonical counts remained unchanged.
+- The post-restart search exposed and repaired a wrapper fallback bug: supported
+  GCP KB commands now fail closed on Cloud SQL errors instead of silently using
+  a different local database. Full repository tests pass `1240/1240`.
 - Persistent GCP `teleo_canonical` is still the older staging copy measured at
   `52,164` rows and `26` proposals. No promotion or production cutover occurred.
 - Fast snapshot restore is now proven. Full reconstruction from every original
@@ -28,6 +38,7 @@ Newest GCP database-first artifacts:
 - `gcp-db-first-parity-current.json`
 - `gcp-db-first-blind-claim-current.json`
 - `gcp-db-first-cleanup-current.json`
+- `gcp-db-first-live-deploy-restart-current.json`
 
 ## Live Addendum - 2026-07-14
 

docs/reports/leo-working-state-20260709/gcp-db-first-live-deploy-restart-current.json

@@ -0,0 +1,125 @@
+{
+  "artifact": "gcp_db_first_live_deploy_restart",
+  "generated_at_utc": "2026-07-14T10:50:13Z",
+  "status": "pass",
+  "source": {
+    "database_first_merge_commit": "08371bf2a0461536c112e1235a7229b52ed270cc",
+    "database_first_pull_request": 144,
+    "fail_closed_wrapper_commit": "8451c51",
+    "repository": "living-ip/teleo-infrastructure"
+  },
+  "deployment": {
+    "compute": "teleo-prod-1",
+    "service": "leoclean-gcp-prod-parallel.service",
+    "database": "teleo_canonical",
+    "database_classification": "GCP staging; not promoted or cut over",
+    "files": {
+      "cloudsql_memory_tool.py": {
+        "before_sha256": "f4396298405b1cbc92590e148b6272c66bb0e8461178edc3ffcfd02bfed69ab4",
+        "after_sha256": "7d2074a0fc5f0d48fbf8f7905a72ead8f2696c86a041fa43e078fff5500baa51",
+        "owner": "teleo:teleo",
+        "mode": "0755"
+      },
+      "teleo-kb": {
+        "before_sha256": "b988079ff2f23b962c40d4c6c9f20c0b85a7bdb0ee675ae146cf36f18ea6529a",
+        "after_sha256": "63cf36cc5840f51a832e7728a4e818d981e106629e44a9f44b7fe8fed6f8a46c",
+        "owner": "teleo:teleo",
+        "mode": "0755"
+      },
+      "teleo-kb-bridge/SKILL.md": {
+        "before_sha256": "523bf6975523c7ea94c043b9e60be3375b122860b951f78ad13dad10fe822de8",
+        "after_sha256": "703fe9aa681ab3aa88dffa4b8c9cf2282329fe6c8f9414ad2d675fb0599679b2",
+        "owner": "teleo:teleo",
+        "mode": "0644"
+      }
+    }
+  },
+  "restart": {
+    "before": {
+      "ActiveState": "active",
+      "SubState": "running",
+      "MainPID": "148735",
+      "NRestarts": "0",
+      "ExecMainStartTimestamp": "Thu 2026-07-09 07:00:04 UTC"
+    },
+    "after": {
+      "ActiveState": "active",
+      "SubState": "running",
+      "MainPID": "304036",
+      "NRestarts": "0",
+      "ExecMainStartTimestamp": "Tue 2026-07-14 10:40:17 UTC"
+    },
+    "controlled_restart_completed": true,
+    "rollback_used": false
+  },
+  "integration_regression": {
+    "detected_live": true,
+    "symptom": "The old auto-mode wrapper used a full receipted status call as a 30-second health probe; the stronger read exceeded that bound and the wrapper fell through to a different local tool.",
+    "repair": "Supported GCP knowledge commands now route directly to Cloud SQL and fail closed on errors instead of changing databases.",
+    "regression_tests": 2,
+    "full_repository_tests": {
+      "passed": 1240,
+      "failed": 0
+    }
+  },
+  "post_restart_status": {
+    "backend": "cloudsql:teleo-pgvector-standby/teleo_canonical/public+kb_stage primary; teleo_kb/teleo_restore fallback",
+    "audit_fallback_available": true,
+    "canonical_counts": {
+      "claims": 1837,
+      "sources": 4145,
+      "claim_edges": 4916,
+      "claim_evidence": 4670,
+      "kb_proposals": 26
+    },
+    "retrieval_receipt": {
+      "schema": "livingip.teleoKbRetrievalReceipt.v1",
+      "semantic_context_sha256": "a428a88de299d9a20a11d614f1edf7b41cbbe78ec5bb7ff4455b6de525252641",
+      "artifact_state_sha256": "4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945",
+      "read_consistency_status": "stable_content_across_wal_change_retry",
+      "attempts": 2,
+      "database": "teleo_canonical"
+    }
+  },
+  "post_restart_search": {
+    "query": "AI sandbagging M&A liability",
+    "command": "search",
+    "backend": "cloudsql:teleo-pgvector-standby/teleo_canonical/public+kb_stage",
+    "expected_claim_found": "2a7ae257-d01d-46f4-b813-63f81bb9c7c7",
+    "expected_sources": [
+      {
+        "id": "15740795-ecc6-40fa-9a01-3d6bc7c54f79",
+        "hash": "341a7685cd2909ae821d8e464fe0d4fdfbb0cadccf546ccfd4ccaf037aa9f646"
+      },
+      {
+        "id": "261c3532-fa32-47d8-a5b5-6cc45035c267",
+        "hash": "9fcd40191449fe0843178e08120fb3c86fc2b35633594aee60e747ca3918cfa5"
+      }
+    ],
+    "retrieval_receipt": {
+      "schema": "livingip.teleoKbRetrievalReceipt.v1",
+      "semantic_context_sha256": "9d2b81918c563a5703794e7e1c52bcafdfc3a4de917eb7f31f36f33f0538ef92",
+      "artifact_state_sha256": "d473c2ab362021dcf0174a4ed4aedfcd95acd705e05b63eaa3f2ee4a9b53bde7",
+      "read_consistency_status": "stable_content_across_wal_change_retry",
+      "attempts": 2,
+      "database": "teleo_canonical"
+    }
+  },
+  "safety": {
+    "telegram_message_sent": false,
+    "canonical_database_write_attempted": false,
+    "database_promoted_or_cut_over": false,
+    "public_cloud_sql_access_enabled": false,
+    "post_restart_model_turn_run": false
+  },
+  "claim_ceiling": {
+    "proven": "Merged database-first helper and skill deployment, fail-closed Cloud SQL routing, controlled GCP service restart, and post-restart live status/search receipts.",
+    "not_proven": [
+      "post-restart model reasoning turn",
+      "Telegram-visible delivery",
+      "canonical proposal apply",
+      "production promotion or cutover",
+      "continuous VPS-to-GCP replication"
+    ]
+  }
+}

docs/reports/leo-working-state-20260709/gcp-db-first-working-leo-20260714.md

@@ -94,6 +94,27 @@ remaining target databases. The disabled rollback database still has zero
 connections. The live GCP Leo service remained PID `148735`, active/running,
 with `NRestarts=0` throughout restore, model replay, and cleanup.
 
+### 5. The merged GCP path survived deployment and restart
+
+PR `#144` merged the database-first helper and skill. Their exact reviewed
+hashes were installed on `teleo-prod-1`, then
+`leoclean-gcp-prod-parallel.service` was intentionally restarted. It returned
+active/running with a new PID (`304036`) and the deployed hashes remained in
+place.
+
+The first post-restart search caught one real integration regression: the old
+wrapper used a full `status` read as a 30-second health probe. Stronger receipts
+made that probe exceed its bound, after which `auto` mode could silently select
+a different local tool. The wrapper now routes every supported GCP knowledge
+command directly to Cloud SQL and fails closed on errors. Two regression tests
+cover direct routing and no fallback.
+
+A corrected live search then found the expected sandbagging claim plus both
+expected source rows and hashes from persistent `teleo_canonical`. A subsequent
+status read returned the same `1,837` claims, `4,145` sources, `4,916` edges,
+`4,670` evidence rows, and `26` proposals as before deployment. Both commands
+returned stable retrieval receipts. No Telegram message or DB write occurred.
+
 ## VPS Versus GCP
 
 | Surface | Current result | Meaning |
@@ -119,6 +140,7 @@ rows as if they were source-code branches.
 | `gcp-db-first-parity-current.json` | `0173ee6707016e8412e6dd4326d61f71b6ef862bbc5b819079d62680676729f2` | Schema, row, role, extension, and performance parity passed |
 | `gcp-db-first-blind-claim-current.json` | `8a7cc3c1814eb385e858f12ef7579cd4524f37886c03fc6d9c61624b6aff2a52` | Real GCP Hermes reasoning passed with source-bound read receipts |
 | `gcp-db-first-cleanup-current.json` | `cac7e34f45653fabb696d911b6ccc921d3f291bf8cbe05a429d757e717dcafb4` | Clone absent, rollback disabled, service unchanged |
+| `gcp-db-first-live-deploy-restart-current.json` | `d70fdbac859b8c98d557f4df900139a93e6009e12f7b80c9adbd69a5655b9df7` | Merged deploy, fail-closed routing, restart, and live status/search readback passed |
 
 ## What This Unlocks
 
@@ -141,6 +163,9 @@ rows as if they were source-code branches.
    and repository artifacts rather than restored from a database snapshot.
 4. VPS changes continuously replicate to GCP or GCP has been formally promoted.
 5. Broad reliability across many unseen claim domains and repeated trials.
+6. A model reasoning turn after the controlled GCP restart. The post-restart
+   proof in this report is the real service plus live `teleo-kb` status/search
+   path, not a new paid model response.
 
 The next product-level proof is one natural Telegram challenge that reaches
 the same database-grounded reasoning, followed by one explicitly approved

hermes-agent/leoclean-bin/teleo-kb

@@ -1,12 +1,11 @@
 #!/usr/bin/env bash
 set -euo pipefail
 
-PROFILE_DIR=/home/teleo/.hermes/profiles/leoclean
+PROFILE_DIR=${HERMES_HOME:-/home/teleo/.hermes/profiles/leoclean}
 KB_TOOL="$PROFILE_DIR/bin/kb_tool.py"
 CLOUDSQL_TOOL="$PROFILE_DIR/bin/cloudsql_memory_tool.py"
 MODE=${TELEO_KB_MODE:-auto}
 COMMAND=${1:-}
-CLOUDSQL_STATUS_TIMEOUT=${TELEO_CLOUDSQL_STATUS_TIMEOUT_SECONDS:-30}
 
 cloudsql_supported() {
   case "$COMMAND" in
@@ -45,12 +44,11 @@ case "$MODE" in
     exec python3 "$KB_TOOL" "$@"
     ;;
   auto)
-    if [ -x "$CLOUDSQL_TOOL" ] && command -v psql >/dev/null 2>&1 && command -v gcloud >/dev/null 2>&1; then
+    # Canonical commands must fail closed on Cloud SQL errors. Falling through
-      if cloudsql_supported; then
+    # to a different local database would make source-of-truth behavior depend
-        if timeout "${CLOUDSQL_STATUS_TIMEOUT}s" python3 "$CLOUDSQL_TOOL" status --format json --redacted >/dev/null 2>&1; then
+    # on health-check latency.
-          run_cloudsql "$@"
+    if [ -x "$CLOUDSQL_TOOL" ] && command -v psql >/dev/null 2>&1 && command -v gcloud >/dev/null 2>&1 && cloudsql_supported; then
-        fi
+      run_cloudsql "$@"
-      fi
     fi
     if command -v docker >/dev/null 2>&1; then
       exec python3 "$KB_TOOL" --local "$@"

tests/test_hermes_leoclean_kb_bridge_source.py

@@ -7,6 +7,7 @@ import contextlib
 import importlib.util
 import io
 import json
+import os
 import re
 import subprocess
 from pathlib import Path
@@ -66,6 +67,76 @@ def test_cloudsql_wrapper_supports_expected_review_gated_commands() -> None:
     assert "canonical apply" in cloudsql_text.lower()
 
 
+def _install_wrapper_fixture(tmp_path: Path) -> tuple[Path, dict[str, str]]:
+    profile = tmp_path / "profile"
+    bin_dir = profile / "bin"
+    bin_dir.mkdir(parents=True)
+    wrapper = bin_dir / "teleo-kb"
+    wrapper.write_bytes((BRIDGE_DIR / "teleo-kb").read_bytes())
+    wrapper.chmod(0o700)
+    cloudsql = bin_dir / "cloudsql_memory_tool.py"
+    cloudsql.write_text(
+        "import json, sys\nprint(json.dumps({'backend': 'cloudsql', 'argv': sys.argv[1:]}))\n",
+        encoding="utf-8",
+    )
+    cloudsql.chmod(0o700)
+    local = bin_dir / "kb_tool.py"
+    local.write_text(
+        "import json, sys\nprint(json.dumps({'backend': 'local', 'argv': sys.argv[1:]}))\n",
+        encoding="utf-8",
+    )
+    local.chmod(0o700)
+    fake_bin = tmp_path / "fake-bin"
+    fake_bin.mkdir()
+    for name in ("psql", "gcloud"):
+        executable = fake_bin / name
+        executable.write_text("#!/usr/bin/env bash\nexit 0\n", encoding="utf-8")
+        executable.chmod(0o700)
+    env = {
+        **os.environ,
+        "HERMES_HOME": str(profile),
+        "TELEO_KB_MODE": "auto",
+        "PATH": f"{fake_bin}:{os.environ['PATH']}",
+    }
+    return wrapper, env
+
+
+def test_cloudsql_wrapper_auto_mode_routes_supported_reads_without_slow_probe(tmp_path: Path) -> None:
+    wrapper, env = _install_wrapper_fixture(tmp_path)
+
+    completed = subprocess.run(
+        [str(wrapper), "search", "AI sandbagging M&A liability", "--format", "json"],
+        text=True,
+        capture_output=True,
+        check=False,
+        env=env,
+    )
+
+    assert completed.returncode == 0, completed.stderr
+    result = json.loads(completed.stdout)
+    assert result == {
+        "backend": "cloudsql",
+        "argv": ["search", "AI sandbagging M&A liability", "--format", "json"],
+    }
+
+
+def test_cloudsql_wrapper_auto_mode_never_falls_back_after_supported_command_failure(tmp_path: Path) -> None:
+    wrapper, env = _install_wrapper_fixture(tmp_path)
+    cloudsql = Path(env["HERMES_HOME"]) / "bin" / "cloudsql_memory_tool.py"
+    cloudsql.write_text("raise SystemExit(23)\n", encoding="utf-8")
+
+    completed = subprocess.run(
+        [str(wrapper), "status", "--format", "json"],
+        text=True,
+        capture_output=True,
+        check=False,
+        env=env,
+    )
+
+    assert completed.returncode == 23
+    assert "local" not in completed.stdout
+
+
 def test_bridge_edge_proposals_stage_strict_apply_payloads() -> None:
     cloudsql_text = (BRIDGE_DIR / "cloudsql_memory_tool.py").read_text()
     vps_text = (BRIDGE_DIR / "kb_tool.py").read_text()

tests/test_repo_skill_pack.py

@@ -79,8 +79,11 @@ def test_gcp_skill_uses_current_operator_access_and_parity_truth() -> None:
         "gcp-db-first-parity-current.json",
         "gcp-db-first-blind-claim-current.json",
         "gcp-db-first-cleanup-current.json",
+        "gcp-db-first-live-deploy-restart-current.json",
         "18/18",
         "6/6",
+        "PID `304036`",
+        "fail closed",
         "invalid_target",
         "older staging copy",
     ):