# KB Restore / Replication Runbook

This runbook is for proving Living IP KB/database redundancy on GCP.

Current source reality:

- canonical runtime DB: `/opt/teleo-eval/pipeline/pipeline.db`
- engine: SQLite WAL
- canonical Leo files:
  - `/opt/teleo-eval/workspaces/main/agents/leo`
  - `/opt/teleo-eval/workspaces/research-leo/agents/leo`
  - `/opt/teleo-eval/agent-state`

The standby target already exists:

- project: `teleo-501523`
- instance: `teleo-pgvector-standby`
- database: `teleo_kb`
- region: `europe-west6`
- network: `teleo-staging-net`
- private IP: `10.61.0.3`
- admin password secret: `gcp-teleo-pgvector-standby-postgres-password`

Do not call this redundancy complete until source data has been restored or replicated and queried from GCP.

## Source Backup Canary

Create a consistent source backup without stopping the VPS service:

```bash
ops/backup_vps_sqlite_kb.sh
```

The script:

- uses SQLite `.backup` against `/opt/teleo-eval/pipeline/pipeline.db`;
- compresses and hashes the backup on the VPS;
- archives Leo/KB files while excluding `secrets` and logs;
- copies both artifacts locally;
- verifies SHA-256 matches;
- runs `PRAGMA integrity_check` on a local restored SQLite copy;
- records proof under `outputs/gcp-infra-hardening-20260707/proofs/`.

This proves source exportability and local restore integrity. It does not prove GCP DB redundancy until a GCP restore/import/query canary also passes.

## Required Proof

A successful restore or replication canary must retain:

- source dataset identity:
  - source host or dump artifact;
  - dump timestamp or replication slot timestamp;
  - source schema/database name.
- transfer proof:
  - dump object path in a versioned bucket, or logical replication subscription details;
  - row/table counts before import where available.
- target proof:
  - `teleo-pgvector-standby` readback;
  - `teleo_kb` database readback;
  - extension readback for `vector` if the restored schema needs pgvector;
  - representative query readback for core KB tables.
- failure boundary:
  - exact missing secret, source access, schema incompatibility, extension issue, or import error.

## One-Shot SQLite Export / GCP Restore Path

Use this while the canonical DB remains SQLite on the VPS and we need a GCP restore drill.

1. Run `ops/backup_vps_sqlite_kb.sh`.
2. Upload the resulting SQLite backup and Leo/KB tarball to a versioned GCS bucket such as `gs://teleo-501523-prod-backups/kb-dumps/`.
3. Convert or replay the SQLite data into Cloud SQL with an explicit migration script. Do not run blind string rewrites against the SQLite dump.
4. Install required extensions on Cloud SQL:

```sql
create extension if not exists vector;
```

5. From a trusted VPC runtime or Cloud SQL connector path, run readbacks:

```sql
select current_database();
select extname, extversion from pg_extension where extname = 'vector';
select schemaname, tablename from pg_tables where schemaname not in ('pg_catalog', 'information_schema') order by 1, 2 limit 50;
```

6. Retain the SQLite backup hash, GCS object generation, import/conversion operation, query output, and row-count sample.

## Logical Replication Path

Use this only if the canonical source becomes Postgres or a Postgres mirror exists. SQLite cannot be logically replicated into Cloud SQL Postgres without an intermediate conversion/sync layer.

Required source privileges:

- replication-capable source user;
- publication over the intended schemas/tables;
- network path from GCP to source, or source-to-GCP path through an approved proxy/tunnel.

Required target steps:

```sql
create extension if not exists vector;
create subscription <subscription_name>
connection '<redacted source connection string>'
publication <publication_name>;
```

Retain only redacted connection metadata. Do not commit or paste credentials.

## Current Blocker

As of this run, GCP has the standby target and a repeatable SQLite/KB export script, but it does not have an approved GCP upload/restore credential in the current local session, nor a retained Cloud SQL import/conversion/query proof. That is why the readiness checker still reports:

- `kb_source_restore_access = blocked`
- `kb_restore_or_replication = blocked`

The next real canary is:

source SQLite/KB backup -> upload to versioned GCS bucket -> convert/import into `teleo_kb` -> install/verify `vector` if needed -> run representative KB queries on GCP -> retain proof.