204 lines
9.5 KiB
Markdown
204 lines
9.5 KiB
Markdown
---
|
|
name: teleo-kb-db-change-workflow
|
|
description: Use for Teleo document-to-claim composition, proposal normalization, separate review/apply authority, isolated apply canaries, row-level proof, rollback, and production-apply gating.
|
|
---
|
|
|
|
# Teleo KB DB Change Workflow
|
|
|
|
## Working Target
|
|
|
|
Turn a source document, post, or operator correction into source-bound evidence,
|
|
atomic claims, graph links, and a reviewable proposal; then move an approved
|
|
change into exact canonical rows only through a guarded apply transaction.
|
|
|
|
Composition and application are separate capabilities. A synthetic apply test
|
|
does not prove source ingestion, and a staged source extraction does not prove
|
|
canonical application.
|
|
|
|
## State Model
|
|
|
|
- `pending_review`: staged, not approved, canonical rows unchanged.
|
|
- `approved`: reviewed intent exists, but `applied_at` may still be null.
|
|
- `applied`: guarded transaction and canonical postflight completed.
|
|
- `packet_ready_not_executed`: rehearsed artifacts exist; production is
|
|
unchanged.
|
|
|
|
Never infer `applied` from chat text, a packet, a clone, or `status=approved`.
|
|
|
|
## Composition Contract
|
|
|
|
A useful composed change must retain:
|
|
|
|
1. immutable source locator, captured bytes or excerpt, and SHA-256 binding;
|
|
2. atomic claim body plus type, confidence, tags, and provenance metadata;
|
|
3. exact evidence excerpt linked to both claim and source;
|
|
4. graph edges, conflict/update/supersession candidates, and uncertainty;
|
|
5. duplicate and existing-row search before staging;
|
|
6. a proposal payload that accounts for every intended canonical row.
|
|
|
|
Do not flatten strategy, governance, concept maps, identity, and reasoning tools
|
|
into generic claim rows merely because the input document discusses them.
|
|
|
|
## V3 Source-To-Rebuild Route
|
|
|
|
For new reconstruction work, use this order:
|
|
|
|
1. Run `ops/run_local_corpus_knowledge_rebuild.py` to retain exact UTF-8
|
|
snapshots and section work items. Its output contains no semantic claims.
|
|
2. For a large corpus, run `ops/shard_corpus_extraction_work.py` to create
|
|
deterministic exact-once work partitions. Sharding schedules extraction; it
|
|
does not read source bodies or produce claims.
|
|
3. Have a separately pinned extractor produce an explicit
|
|
`livingip.kbSourceExtractionManifest.v3`. Every evidence quote must bind to
|
|
exact UTF-8 byte offsets in the retained extraction.
|
|
4. Run `scripts/compile_kb_source_packet.py` with the original artifact, exact
|
|
extracted text, manifest, and extractor specification. The result is
|
|
build-only and remains `pending_review`.
|
|
5. On a disposable PostgreSQL clone, stage with
|
|
`scripts/stage_normalized_proposal.py --compiled-v3-bundle ...`, then render
|
|
the exact rows with `scripts/kb_proposal_review_packet.py`.
|
|
6. Record a real review separately with `scripts/approve_proposal.py` and exact
|
|
`--reviewed-by m3taversal`. Never use test-entered review metadata as
|
|
evidence that `m3taversal` actually reviewed a production packet.
|
|
7. Apply only the exact UUID with `scripts/apply_worker.py`, `--enable`,
|
|
`--contract-version 3`, and `--proposal-id ...`; refuse rather than draining
|
|
another approved proposal from the queue.
|
|
8. Export the applied transition with
|
|
`scripts/export_kb_transition_replay_material.py`, assemble ordered V3
|
|
materials with `ops/build_v3_genesis_ledger_bundle.py`, and independently
|
|
replay with `ops/run_local_genesis_ledger_rebuild.py`.
|
|
|
|
This repository does not yet provide a general PDF/office extractor or an
|
|
automatic corpus-wide semantic extractor. For binary originals, retain their
|
|
exact bytes and pin the separately produced UTF-8 extraction plus extractor
|
|
specification before step 3.
|
|
|
|
Run the deterministic source-to-proposal canary:
|
|
|
|
```bash
|
|
.venv/bin/python scripts/run_leo_local_ingestion_proposal_canary.py \
|
|
--fixture fixtures/working-leo/document-ingestion-v1.json \
|
|
--output /tmp/leo-local-ingestion-proposal-canary-current.json
|
|
```
|
|
|
|
Run the complete disposable V3 source-to-clean-rebuild lifecycle:
|
|
|
|
```bash
|
|
.venv/bin/python -m pytest -q tests/test_v3_source_rebuild_lifecycle_postgres.py
|
|
```
|
|
|
|
This lifecycle uses a synthetic pinned extractor manifest and synthetic
|
|
operator-supplied review metadata. It proves the local machinery and exact
|
|
replay boundary, not semantic extraction quality, real human approval, or a
|
|
live database mutation.
|
|
|
|
Require the source, claim, evidence, and proposal UUIDs to link exactly; claim
|
|
and evidence text must be exact substrings of the source; status must remain
|
|
`pending_review`; the container must use network `none` and leave no volume or
|
|
container behind.
|
|
|
|
## Canonical Apply Contract
|
|
|
|
The strict `approve_claim` v2 contract remains supported for its retained
|
|
proofs. V2/V3 payloads may create the supported canonical rows for their exact
|
|
contract version; V3 additionally carries explicit provenance, access,
|
|
assessment, and admission fields.
|
|
|
|
The shared row families include:
|
|
|
|
- `public.claims`
|
|
- `public.sources`
|
|
- `public.claim_evidence`
|
|
- `public.claim_edges`
|
|
- `public.reasoning_tools`
|
|
|
|
The lifecycle is split across:
|
|
|
|
- `scripts/compile_kb_source_packet.py`: hash-bound source plus V3 semantic
|
|
manifest to a deterministic build-only bundle;
|
|
- `scripts/stage_normalized_proposal.py`: revalidate and stage an exact bundle
|
|
only on the named disposable database;
|
|
- `scripts/kb_proposal_review_packet.py`: read-only review of exact proposed
|
|
canonical rows;
|
|
- `scripts/kb_proposal_normalize.py`: fail-closed rich intent to strict payload;
|
|
- `scripts/approve_proposal.py`: `kb_review` approves the exact type/payload;
|
|
- `scripts/apply_proposal.py`: operator-only `kb_apply` writes and verifies;
|
|
- `scripts/apply_worker.py`: exact-ID, one-at-a-time V2/V3 worker wrapper;
|
|
- `scripts/export_kb_transition_replay_material.py`: read-only applied-transition
|
|
exporter;
|
|
- `ops/build_v3_genesis_ledger_bundle.py`: V3-only ordered ledger assembler;
|
|
- `scripts/kb_apply_prereqs.sql`: roles, immutable approval row, gate functions,
|
|
ownership, indexes, and ACLs;
|
|
- `scripts/run_approve_claim_isolated_container_canary.sh`: disposable
|
|
lifecycle plus optional live-readonly source/service checks.
|
|
|
|
## Authority Invariants
|
|
|
|
1. `kb_review` and `kb_apply` are separate `NOINHERIT` login roles.
|
|
2. `kb_gate_owner` is `NOLOGIN` and owns immutable approval/gate functions.
|
|
3. Review binds DB role, reviewer, type, full payload, timestamp, and note.
|
|
4. Apply locks and compares that snapshot before canonical writes.
|
|
5. Payload-controlled fields and exact table deltas are verified.
|
|
6. Existing evidence/edge semantic mismatches roll back the transaction.
|
|
7. Replay, stale payload, source-hash drift, and direct privilege probes fail.
|
|
8. `kb_apply` remains a trusted operator-only canonical writer, not autonomous
|
|
chat write authority.
|
|
|
|
## Operator Path
|
|
|
|
1. Read the proposal, source bytes/hash, and canonical target rows.
|
|
2. Normalize rich intent; reject lossy, duplicate, or unsupported mappings.
|
|
3. Run focused tests:
|
|
|
|
```bash
|
|
.venv/bin/python -m pytest -q \
|
|
tests/test_kb_proposal_normalize.py \
|
|
tests/test_approve_proposal.py \
|
|
tests/test_apply_proposal.py \
|
|
tests/test_kb_apply_prereqs.py
|
|
```
|
|
|
|
4. Run the generic and real-packet disposable canaries.
|
|
5. Require exact before/after projections, deltas, source hashes, approval row,
|
|
applied timestamp, service state, rollback, and cleanup.
|
|
6. Treat source checkout deployment, permission migration, worker enablement,
|
|
and production proposal application as separate windows.
|
|
|
|
## Current Proof - 2026-07-12
|
|
|
|
- Deterministic source composition passes all `13` checks and links one source,
|
|
one extracted claim, one evidence row, and one `pending_review` proposal. It
|
|
proves the retained fixture path, not arbitrary-source model extraction.
|
|
Arbitrary production document/tweet ingestion is not proven.
|
|
- The isolated approved-change lifecycle creates exact deltas of `2` claims,
|
|
`2` sources, `2` evidence rows, `1` edge, and `1` reasoning tool, stamps the
|
|
proposal applied, rolls back, and leaves no container or volume.
|
|
- The earlier generic and Helmer v3 source-bound receipts remain `37/37`; the
|
|
newer lifecycle receipt adds stricter service/source isolation and cleanup.
|
|
- A VPS live-readonly T3 clone run passed with unchanged canonical counts and
|
|
unchanged gateway PID. Production was not applied.
|
|
- The broader live-VPS full-data source-composition checkpoint passes `34/34`:
|
|
new hash-bound document/post, conflicting atomic claims, exact source/evidence
|
|
links, strict proposal, separated approval/apply, restarted discovery, graph
|
|
reasoning, and cleanup. This is isolated clone proof, not Telegram delivery
|
|
or production mutation.
|
|
- PR #86 merged the harness/operator fixes. VPS auto-deploy synchronized source
|
|
without restarting Leo; the gateway remained PID `2403328`, `NRestarts=0`.
|
|
- Current VPS canonical counts remain claims `1837`, sources `4145`, evidence
|
|
`4670`, edges `4916`, proposals `26`.
|
|
|
|
Read:
|
|
|
|
- `docs/reports/leo-working-state-20260709/approve-claim-clone-canary-current.json`
|
|
- `docs/reports/leo-working-state-20260709/leo-source-composition-clone-checkpoint-current.json`
|
|
- `docs/reports/leo-working-state-20260709/leo-v3-document-source-lifecycle-current.md`
|
|
- `docs/reports/leo-working-state-20260709/source-document-compiler-canary-20260713.md`
|
|
|
|
## Claim Ceiling
|
|
|
|
Leo has strong isolated proof for source-bound composition, guarded canonical
|
|
application, restarted discovery, and graph reasoning. Production still has
|
|
three approved-but-unapplied legacy packets and fourteen pending-review
|
|
proposals. No broad autonomous production apply authority exists, and no
|
|
production rich packet should be applied without exact retained authorization,
|
|
operator identity, postflight, regression, rollback, and cleanup evidence.
|