teleo-infrastructure/.agents/skills/teleo-gcp-parity-ops/SKILL.md
2026-07-15 22:45:39 +02:00

266 lines
14 KiB
Markdown

---
name: teleo-gcp-parity-ops
description: Use for passwordless Teleo GCP VM access, private Cloud SQL canonical parity, GCP Leo runtime readback, m3taversal no-send replay, rollback, and cleanup without collapsing VPS proof into GCP proof.
---
# Teleo GCP Parity Ops
## Working Target
Restore a copy of the VPS canonical Leo database to GCP, prove exact schema,
row, role membership, ownership, ACL, extension, performance, and
private-connectivity parity, then run the
real GCP Leo read/reasoning path without Telegram sends or DB mutation.
The required tier is T3: a live private GCP staging restore, readback,
reasoning, compute attestation, and cleanup receipt. Local restore proof or an
access-gate report does not satisfy this tier.
## Operator Paths
The direct alias was passwordless and live-verified on 2026-07-14:
```bash
ssh teleo-gcp-staging
```
It uses `~/.ssh/google_compute_engine` for the configured operator,
disables password and keyboard-interactive authentication, and does not store a
Google password. That is historical proof, not current reachability. The route
remains firewall-source-dependent. On
2026-07-15, both direct SSH and the private TCP route timed out; verify
`ssh -o BatchMode=yes teleo-gcp-staging true` before a long run and fail closed
if it does not return successfully.
The intended secondary path is `.github/workflows/gcp-iap-operator.yml`, using
short-lived GitHub OIDC, IAP, OS Login, and fixed reviewed operations. It is
merged but not bootstrapped: the current 2026-07-15 check still lacks the
reviewed operator WIF/IAP authority. Provider `teleo-iap-operator` is absent,
disabled, or deleted (`invalid_target`), and the available artifact identity is intentionally not
a Cloud SQL/Compute operator. Do not call this path working until a live `status` run passes
with the intended operator identity.
Historically verified and intended target; reverify before any new lifecycle:
- project `teleo-501523`;
- VM `teleo-prod-1` in `europe-west6-a`;
- Cloud SQL `teleo-pgvector-standby`, PostgreSQL 16.14;
- private endpoint `10.61.0.3:5432`, public IP disabled, TLS required;
- restore identity `sa-teleo-restore-drill@teleo-501523.iam.gserviceaccount.com`;
- service `leoclean-gcp-prod-parallel.service`.
Direct libpq connections must use `sslmode=verify-full`, a reviewed DNS name
covered by the server certificate, and a private regular CA file. The private
IP is `hostaddr`, not the certificate identity. Never downgrade to
`sslmode=require` or accept an arbitrary `*.iam.gserviceaccount.com` identity.
Never print the Cloud SQL or Google password. On the VM, use the attached service account
and Secret Manager through reviewed wrappers or a short-lived environment
variable, then unset it.
## Two Different Databases
- Canonical collective knowledge remains VPS PostgreSQL database `teleo`, with
`public.*` and `kb_stage.*` tables. Persistent Cloud SQL database
`teleo_canonical` is a staging copy and has not been promoted.
- Hermes conversation continuity is `state.db` plus session JSONL files. The
`leoclean-cloudsql-memory-sync.service` snapshot path copies this runtime
memory; it does not populate or prove canonical claims, sources, evidence,
edges, or proposals.
Do not describe a passing Hermes memory sync as canonical KB parity.
## Retained Historical Proof - 2026-07-14
- The newest captured VPS database has `39` tables and `52,167` rows, including
claims `1837`, sources `4145`, claim evidence `4670`, claim edges `4916`, and
proposals `29`.
- A disposable private-TLS GCP clone restored that snapshot with exact
`39/39`-table and `52,167/52,167`-row parity. Rowsets, schema objects, roles,
extensions, constraints, and performance checks had zero mismatches.
- A real no-send GCP Hermes turn received an ID-free claim challenge, performed
`search`, `show`, `evidence`, and `edges`, retrieved the expected claim and
both source rows, and passed `18/18` runtime checks plus `6/6` reasoning
outcomes. It did not send Telegram or write the DB.
- `status` and zero-hit search work on a canonical-only clone without the
optional `teleo_restore` audit schema. All read commands emit deterministic
retrieval receipts.
- The generated clone was deleted. The retained rollback database remains
connection-disabled with zero sessions. The GCP gateway remained PID
`148735`, `NRestarts=0`, active/running throughout that bounded experiment.
- PR `#144` merged the reviewed helper/skill. After deployment, the service
survived a controlled restart and is active/running at PID `304036`, start
time `2026-07-14 10:40:17 UTC`. Live post-restart `status` and `search`
returned Cloud SQL retrieval receipts with unchanged canonical counts.
- A live regression showed the old wrapper could time out its stronger status
probe and fall through to a different local tool. Supported GCP KB commands
now route directly to Cloud SQL and fail closed on errors; two behavioral
tests enforce that invariant.
- Persistent GCP `teleo_canonical` remains the older staging copy measured at
`52,164` rows and `26` proposals. It has not been promoted or cut over.
That historical clone predates the expanded ownership/ACL and independently
authenticated receipt contract. It remains useful historical row/schema proof,
but it is not current T3 proof under this skill.
## Open Least-Privilege Candidates
PR #148, `Scope GCP Leo runtime to least-privilege Cloud SQL access`, is open as
of the 2026-07-15 skill-pack reconciliation. Draft PR #162, `Harden GCP Leo
runtime least-privilege proof`, is its current hardening successor. They propose
scoped runtime roles, secret access, fail-closed Cloud SQL behavior, deployment
rollback, and positive plus negative permission checks. Those branch files and
proposed live outcomes are candidate evidence only. Before using them, run:
```bash
gh pr view 148 --json state,mergedAt,mergeCommit,headRefName,url
gh pr view 162 --json state,mergedAt,mergeCommit,headRefName,url
```
Until a successor is merged and its runtime receipt passes, use only paths present on
canonical `main`, keep persistent GCP classified as staging, do not promote it,
and do not infer least-privilege cutover from a branch or dry run.
Primary retained proof:
- `docs/reports/leo-working-state-20260709/gcp-db-first-working-leo-20260714.md`
- `docs/reports/leo-working-state-20260709/gcp-db-first-restore-current.json`
- `docs/reports/leo-working-state-20260709/gcp-db-first-parity-current.json`
- `docs/reports/leo-working-state-20260709/gcp-db-first-blind-claim-current.json`
- `docs/reports/leo-working-state-20260709/gcp-db-first-cleanup-current.json`
- `docs/reports/leo-working-state-20260709/gcp-db-first-live-deploy-restart-current.json`
## Required Parity Rows
Track these independently:
1. control-plane project, VM, Cloud SQL, private-IP, and TLS identity;
2. canonical database schema, counts, row hashes, constraints, indexes,
functions, extensions, exact role attributes, connection limits,
`default_transaction_read_only` settings and memberships, owners,
normalized schema/relation/column/function/type ACLs, and performance;
3. GCP service PID/restarts plus live profile and tool hashes;
4. `DC-01` through `DC-06` DB-read readiness;
5. real no-send model replies, strict score, and exact count consistency;
6. no-mutation fingerprints, child/profile cleanup, clone deletion, and rollback
disposition.
## Canonical Restore And Replay
Use:
- `ops/capture_vps_canonical_postgres_snapshot.py` for a single source dump and
manifest, with a required retained `--authorization-ref` and v2 provenance
receipt binding the exported snapshot identity plus dump/manifest hashes; it
requires identical healthy structured source-service states before/after
(`active/running`, positive PID, nonnegative restart count);
- `ops/restore_gcp_generated_postgres_snapshot.py restore --execute` for a
receipt-bound private-TLS restore into a bounded `teleo_clone_*` database;
pass the exact capture `--source-receipt`, and require its live Cloud SQL
control-plane check to prove public IP disabled before clone creation; it
requires the exact restore service account plus `--ssl-server-name` and
`--ssl-root-cert` for certificate-authenticated `verify-full` TLS; it
writes `target-manifest.jsonl`, `restore-receipt.json`, and the generated
`gcp-private-connectivity.json` under
`/var/lib/teleo-gcp-restore-runs/<restore-run-id>/`;
- `ops/restore_gcp_generated_postgres_snapshot.py cleanup --execute` for exact
clone cleanup with live-service and rollback readback;
- `ops/postgres_parity_manifest.sql` for row/catalog/role/performance readback;
- `ops/verify_postgres_parity_manifest.py --scope gcp_staging` for exact parity,
always retaining the canonical output name `gcp-parity.json`;
- `scripts/run_gcp_generated_db_direct_claim_suite.py` for the adapter-free,
read-only, no-send six-response replay against a generated `teleo_clone_*`.
- `scripts/run_gcp_generated_db_blind_claim_canary.py` for the bounded ID-free
claim challenge, source receipt, reasoning, no-write, and cleanup proof.
- `ops/attest_gcp_reasoning_compute.py` on the replay VM after blind reasoning
and before cleanup, binding the reasoning receipt SHA-256 to live GCE metadata
in `reasoning-compute-attestation.json`;
- a second authorized source capture after cleanup, followed by
`ops/verify_vps_canonical_snapshot_delta.py`, to retain the explicit
post-snapshot VPS delta in `source-delta-receipt.json`;
- `ops/verify_gcp_canonical_lifecycle.py` after cleanup to bind the exact source,
current source, source delta, restore, `gcp_staging` parity, ID-free reasoning,
reasoning-compute attestation, and cleanup receipts into one fail-closed
preparation verdict. Caller-writable JSON can never emit T3 from this API;
even an internally valid bundle returns
`prepared_external_attestation_required`, `current_tier=T2`, and
`accepted_by_this_api=false`. Pass `--max-postflight-age-seconds 900` and
`--max-lifecycle-age-seconds 3600`; the latter bounds both cleanup-to-verdict
age and restore-to-postflight span.
With `TARGET_DB`, `RESTORE_RUN_ID`, and `PRIVATE_RUN_DIR` set as shown in the
runbook, run the replay-host attestation after the blind receipt passes and
before the receipt-bound cleanup:
```bash
sudo python3 ops/attest_gcp_reasoning_compute.py \
--reasoning-receipt "${PRIVATE_RUN_DIR}/blind-reasoning-receipt.json" \
--restore-run-id "$RESTORE_RUN_ID" \
--target-db "$TARGET_DB" \
--project teleo-501523 \
--expected-compute-instance teleo-prod-1 \
--expected-compute-zone europe-west6-a \
--expected-private-network projects/teleo-501523/global/networks/teleo-staging-net \
--expected-restore-service-account sa-teleo-restore-drill@teleo-501523.iam.gserviceaccount.com \
--output "${PRIVATE_RUN_DIR}/reasoning-compute-attestation.json"
```
The replay must use the Hermes virtualenv, not system Python:
```bash
/home/teleo/.hermes/hermes-agent/venv/bin/python \
scripts/run_gcp_generated_db_direct_claim_suite.py ...
```
The lifecycle is not complete until the generated clone, temporary profile,
children, transient upload/bundle directories, and any temporary client are
absent. Preserve the private receipt run directory and every useful lifecycle
input, supporting receipt, and verifier output; do not treat that evidence as
transient cleanup. The canonical restore streams through `pg_restore` and
creates no GCS import object.
The source snapshot retains ACL commands while omitting owner commands. The
networkless local preflight can therefore replay captured ACLs after creating
only the exact allowlisted application roles, plus the bounded database-level
grants that `pg_restore` does not apply to an already-created database. The
shared-Cloud-SQL restore still uses `--no-owner` and `--no-privileges`, so GCP
parity must fail until a separately reviewed Cloud-SQL-compatible authorization
replay restores the captured owner/grant semantics. Do not bypass this by
dropping manifest rows or by running the Docker/superuser role bootstrap
blindly on shared Cloud SQL.
Static inspection or a dry run is insufficient for these helpers: verify a new
capture against the live VPS and run the focused tests. A current T3 claim also
requires a separate independently authenticated provider/platform receipt that
the bundle author cannot mint.
## Safety And Claim Ceiling
- Do not send Telegram messages from the GCP parity lane.
- Do not apply, approve, or stage KB changes during read/reasoning replay.
- Do not restart the live GCP gateway for tool-file synchronization unless a
separate restart window is explicitly requested.
- Do not call control-plane inventory, memory sync, route readiness, or a
nominal scorer pass full m3taversal parity.
The strongest claim emitted by the repository lifecycle verifier is T2
structural consistency. Exact DB parity plus real no-send model replies with
truthful counts and cleanup still requires independent T3 attestation. Neither proves Telegram
delivery, GCP canonical mutation, ongoing replication, or production cutover.
## Current Access And Next Action
As of 2026-07-15, direct SSH and private TCP timed out, local `gcloud` for
`billy@livingip.xyz` requires reauthentication, and the intended operator
WIF/IAP authority is unavailable. The artifact-builder WIF identity must not be
expanded or mistaken for database-operator access. Current retained T2/source
and local restore proof does not satisfy the required T3 lifecycle.
The next action after an authorized operator route is restored is: newest
authorized VPS capture -> disposable private `teleo_clone_*` restore ->
generated private-connectivity receipt -> `gcp-parity.json` -> no-send
m3taversal blind reasoning -> live GCE reasoning attestation -> receipt-bound
clone cleanup -> postflight VPS capture/delta -> eight-receipt lifecycle verdict
within the 900/3600-second freshness bounds. Do not promote
`teleo_canonical`, expose Cloud SQL publicly, send Telegram, or mutate canonical
knowledge.