Merge pull request #75 from living-ip/codex/working-leo-live-clone-20260710
Some checks are pending
CI / lint-and-test (push) Waiting to run
Some checks are pending
CI / lint-and-test (push) Waiting to run
Prove guarded Leo source composition on VPS clone
This commit is contained in:
commit
a4462b1fbe
18 changed files with 11412 additions and 70 deletions
|
|
@ -109,6 +109,112 @@ live count/service endpoints without writing them, and independently prove the
|
||||||
container/workdir are absent afterward. Do not install the candidate code into
|
container/workdir are absent afterward. Do not install the candidate code into
|
||||||
the live Leo deploy merely to run this canary.
|
the live Leo deploy merely to run this canary.
|
||||||
|
|
||||||
|
To prove that Leo itself can inspect one lifecycle state through the real
|
||||||
|
`GatewayRunner` while remaining bound to a disposable full-data clone, run the
|
||||||
|
checkpoint on the VPS as `teleo` while that clone exists:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo -u teleo install -d -m 700 /home/teleo/leo-checkpoint-reports
|
||||||
|
sudo -u teleo HOME=/home/teleo \
|
||||||
|
/home/teleo/.hermes/hermes-agent/venv/bin/python \
|
||||||
|
scripts/run_leo_clone_bound_handler_checkpoint.py \
|
||||||
|
--container <disposable-container> \
|
||||||
|
--db teleo \
|
||||||
|
--prompt-id <stable-id> \
|
||||||
|
--prompt "<Cory-style KB question>" \
|
||||||
|
--expected-state approved \
|
||||||
|
--copy-model-auth \
|
||||||
|
--output /home/teleo/leo-checkpoint-reports/leo-clone-bound-checkpoint.json
|
||||||
|
```
|
||||||
|
|
||||||
|
The supplied clone must have Docker label
|
||||||
|
`com.livingip.leo.checkpoint=disposable`, network mode `none`, no published
|
||||||
|
ports, no mount source shared with production, and a PostgreSQL system
|
||||||
|
identifier distinct from production. The harness resolves the name once, pins
|
||||||
|
all operations to the full container ID, and fails if the name is rebound.
|
||||||
|
|
||||||
|
`--copy-model-auth` copies only `auth.json` into a private UUID-scoped profile
|
||||||
|
and binds only the configured provider's env-backed credential in memory. The
|
||||||
|
receipt records provider/variable names, never credential values or hashes. The
|
||||||
|
gateway runs in a dedicated process group with no delivery adapters and only
|
||||||
|
`skills_list`, `skill_view`, and a terminal handler restricted to the temporary
|
||||||
|
clone-bound wrapper's read-only verbs. The terminal subprocess must not inherit
|
||||||
|
the model credential. A timeout must terminate the child process group before
|
||||||
|
the profile is removed.
|
||||||
|
|
||||||
|
The checkpoint must observe at least one successful `teleo-kb` call bound to
|
||||||
|
the supplied container/database, remove the profile, and prove production row
|
||||||
|
counts plus row-content fingerprints, gateway state, and live bridge hashes are
|
||||||
|
unchanged. Every model tool call must be in the allowlist and every KB call must
|
||||||
|
complete successfully for a clean pass. A correct answer after a failed call
|
||||||
|
and retry is recovered behavior, not a clean reliability pass. The checkpoint
|
||||||
|
does not review, approve, apply, restart, post to Telegram, or mutate
|
||||||
|
production.
|
||||||
|
|
||||||
|
After that read-only checkpoint is clean, prove the lower-level
|
||||||
|
pending-to-canonical lifecycle in the same kind of disposable clone:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo -u teleo HOME=/home/teleo \
|
||||||
|
/home/teleo/.hermes/hermes-agent/venv/bin/python \
|
||||||
|
scripts/run_leo_clone_lifecycle_checkpoint.py \
|
||||||
|
--container <disposable-container> \
|
||||||
|
--db teleo \
|
||||||
|
--copy-model-auth \
|
||||||
|
--operator-review \
|
||||||
|
--guarded-apply \
|
||||||
|
--output /home/teleo/leo-checkpoint-reports/leo-clone-lifecycle-checkpoint.json
|
||||||
|
```
|
||||||
|
|
||||||
|
This command gives Leo only one deterministic staging verb. The harness, not
|
||||||
|
Leo, owns review and guarded apply. A pass requires exact structured receipts
|
||||||
|
for conversation memory, pending, approved-but-unapplied, and applied state;
|
||||||
|
exact linked claim/source/evidence rows; an isolated handler reopen using the
|
||||||
|
same persisted session; zero rejected or nonzero terminal calls; and unchanged
|
||||||
|
production DB fingerprints, service state, and live bridge hashes. It is a
|
||||||
|
mutation primitive, not proof that Leo can extract knowledge from arbitrary
|
||||||
|
documents or tweets.
|
||||||
|
|
||||||
|
For the real source-composition checkpoint, first create a fresh full-data
|
||||||
|
clone with the same disposable label/network/mount rules, then install the gate
|
||||||
|
and separated ephemeral clone credentials into a private run directory:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo -u teleo HOME=/home/teleo \
|
||||||
|
/home/teleo/.hermes/hermes-agent/venv/bin/python \
|
||||||
|
scripts/bootstrap_clone_kb_gate.py \
|
||||||
|
--container <fresh-disposable-container> \
|
||||||
|
--db teleo \
|
||||||
|
--output-dir <private-run-directory>
|
||||||
|
```
|
||||||
|
|
||||||
|
The bootstrap must verify both `kb_review` and `kb_apply` logins and emit only
|
||||||
|
credential file paths, never secret values or hashes. Then run:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo -u teleo HOME=/home/teleo \
|
||||||
|
/home/teleo/.hermes/hermes-agent/venv/bin/python \
|
||||||
|
scripts/run_leo_clone_composition_checkpoint.py \
|
||||||
|
--container <fresh-disposable-container> \
|
||||||
|
--db teleo \
|
||||||
|
--copy-model-auth \
|
||||||
|
--operator-review \
|
||||||
|
--guarded-apply \
|
||||||
|
--review-secrets-file <private-run-directory>/kb-review.env \
|
||||||
|
--apply-secrets-file <private-run-directory>/kb-apply.env \
|
||||||
|
--output <private-run-directory>/reports/composition-current.json
|
||||||
|
```
|
||||||
|
|
||||||
|
A pass requires model-driven dedupe search and extraction from supplied source
|
||||||
|
bytes, exact source hashes and excerpts, linked claims/evidence/conflict edges,
|
||||||
|
no row change before staging, deterministic normalization, separated review and
|
||||||
|
apply, exact clone deltas, a new handler process reasoning over exact canonical
|
||||||
|
rows without supplied IDs, session-marker recall, nonce-bound tool receipts,
|
||||||
|
unchanged production DB/service/bridge state, and complete clone/profile/secret
|
||||||
|
cleanup. The retained current receipt is
|
||||||
|
`docs/reports/leo-working-state-20260709/leo-source-composition-clone-checkpoint-current.json`.
|
||||||
|
It is not Telegram-visible proof and does not authorize production apply.
|
||||||
|
|
||||||
## Blocker Format
|
## Blocker Format
|
||||||
|
|
||||||
Do not say "blocked" without:
|
Do not say "blocked" without:
|
||||||
|
|
|
||||||
|
|
@ -27,12 +27,25 @@ A working Leo is a Telegram-facing agent that:
|
||||||
3. Distinguishes `proposed`, `pending_review`, `approved`, `applied`, and `not applied`.
|
3. Distinguishes `proposed`, `pending_review`, `approved`, `applied`, and `not applied`.
|
||||||
4. Does not say an approval changed canonical DB state when only `kb_stage` changed.
|
4. Does not say an approval changed canonical DB state when only `kb_stage` changed.
|
||||||
5. Can answer vague, high-level Cory-style incident prompts without being spoon-fed exact proposal IDs.
|
5. Can answer vague, high-level Cory-style incident prompts without being spoon-fed exact proposal IDs.
|
||||||
6. Can stage concrete KB changes from Telegram with enough structure for review.
|
6. Can compose the KB from a previously unindexed document, URL, or tweet-like
|
||||||
7. Can move approved concrete changes through a guarded apply path when authorized.
|
source: retain a byte/hash-bound source locator, extract atomic claims and
|
||||||
8. Retains Cory's caveats and review notes in source/evidence/proposal rows.
|
exact evidence excerpts, preserve useful metadata, and link every claim to
|
||||||
9. Produces before/after table-level proof and service stability readback.
|
its evidence and source.
|
||||||
10. Survives an intentional `leoclean-gateway.service` restart: active before, active after, canonical KB counts unchanged, and a no-post handler smoke still answers.
|
7. Detects duplicates, conflicts, updates, and insufficient evidence before
|
||||||
11. For a reviewed graph bundle, produces exact payload-controlled row
|
staging; uncertainty must remain visible instead of being normalized into a
|
||||||
|
stronger claim.
|
||||||
|
8. Can stage those concrete KB changes from Telegram with enough structure for
|
||||||
|
human review, without making staged content canonical.
|
||||||
|
9. Can move approved concrete changes through a guarded apply path when authorized.
|
||||||
|
10. Retains Cory's caveats and review notes in source/evidence/proposal rows.
|
||||||
|
11. Reasons over claims, evidence, sources, edges, and open conflicts as a graph,
|
||||||
|
and can explain which rows support or weaken an answer without being given IDs.
|
||||||
|
12. Rebuilds any compiled identity/workspace artifact deterministically from
|
||||||
|
canonical DB rows, reports the source rows and freshness boundary, and does
|
||||||
|
not treat edits to a generated artifact as canonical knowledge.
|
||||||
|
13. Produces before/after table-level proof and service stability readback.
|
||||||
|
14. Survives an intentional `leoclean-gateway.service` restart: active before, active after, canonical KB counts unchanged, and a no-post handler smoke still answers.
|
||||||
|
15. For a reviewed graph bundle, produces exact payload-controlled row
|
||||||
projections, exact table deltas, source-byte binding, and cleanup proof in a
|
projections, exact table deltas, source-byte binding, and cleanup proof in a
|
||||||
disposable runtime before any production permission or apply window.
|
disposable runtime before any production permission or apply window.
|
||||||
|
|
||||||
|
|
@ -46,6 +59,26 @@ A working Leo is a Telegram-facing agent that:
|
||||||
- Old rich proposal packets: `14fa5ecc...`, `ac036c9d...`, and `a64df080...` are not to be silently production-applied.
|
- Old rich proposal packets: `14fa5ecc...`, `ac036c9d...`, and `a64df080...` are not to be silently production-applied.
|
||||||
- Guarded apply proof: both generic and real Helmer v3 receipts pass `37/37` in
|
- Guarded apply proof: both generic and real Helmer v3 receipts pass `37/37` in
|
||||||
disposable PostgreSQL; production Helmer remains unapplied.
|
disposable PostgreSQL; production Helmer remains unapplied.
|
||||||
|
- Clone-bound handler proof: the real VPS `GatewayRunner` can inspect the full
|
||||||
|
current-data disposable clone, discover the Helmer proposal, distinguish
|
||||||
|
`approved` from `applied`, and produce bound bridge-call evidence without a
|
||||||
|
Telegram post or production change. The proof surface must expose no delivery
|
||||||
|
adapters or send tool, restrict terminal execution to clone-bound read-only KB
|
||||||
|
verbs, kill the handler process group on timeout, and compare production
|
||||||
|
row-content fingerprints before and after. Treat open-ended latency and
|
||||||
|
retries as a separate reliability dimension; a correct answer after a failed
|
||||||
|
tool call is recovered behavior, not a clean pass, and one correct answer does
|
||||||
|
not establish a stable pass rate.
|
||||||
|
- Source-composition proof: the real VPS `GatewayRunner` passed `34/34` in a
|
||||||
|
fresh no-send full-data clone. It searched existing knowledge, extracted two
|
||||||
|
hash-bound conflicting claims from a new document and post, staged a strict
|
||||||
|
proposal, preserved an immutable separated approval, applied exact canonical
|
||||||
|
rows, reopened in a new child process, recalled the prior marker, discovered
|
||||||
|
the new proposal/claims without supplied IDs, read exact evidence/source UUIDs
|
||||||
|
and edges, and explained `approved != applied`. Production DB fingerprints,
|
||||||
|
service PID/restarts, and live bridge hashes stayed unchanged; all disposable
|
||||||
|
containers, volumes, profiles, credentials, and run directories were removed.
|
||||||
|
This is clone proof, not Telegram delivery or production apply proof.
|
||||||
|
|
||||||
## Cory Direct Questions
|
## Cory Direct Questions
|
||||||
|
|
||||||
|
|
@ -90,7 +123,15 @@ Leo is not working just because:
|
||||||
|
|
||||||
Those are evidence. The target is Telegram-visible behavior plus DB-state truth plus a guarded path to canonical rows.
|
Those are evidence. The target is Telegram-visible behavior plus DB-state truth plus a guarded path to canonical rows.
|
||||||
|
|
||||||
|
A synthetic one-claim proposal proves only the guarded mutation primitive. It
|
||||||
|
does not prove KB composition. Composition requires a source that was absent at
|
||||||
|
baseline, model-driven extraction into linked source/evidence/claim proposals,
|
||||||
|
dedupe/conflict readback, guarded canonical apply, and an open-ended answer
|
||||||
|
grounded in the new rows after an isolated handler restart.
|
||||||
|
|
||||||
## Proof Files
|
## Proof Files
|
||||||
|
|
||||||
Use `docs/reports/leo-working-state-20260709/current-truth-index.md` for current artifact paths.
|
Use `docs/reports/leo-working-state-20260709/current-truth-index.md` for current artifact paths.
|
||||||
|
Use `docs/reports/leo-working-state-20260709/leo-source-composition-clone-checkpoint-current.md`
|
||||||
|
and its JSON companion for the current source-composition claim ceiling.
|
||||||
For restart survival, use `scripts/collect_leo_restart_survival_proof.py --execute-restart`; expected artifacts are `docs/reports/leo-working-state-20260709/leo-restart-survival-proof-current.json` and `.md`.
|
For restart survival, use `scripts/collect_leo_restart_survival_proof.py --execute-restart`; expected artifacts are `docs/reports/leo-working-state-20260709/leo-restart-survival-proof-current.json` and `.md`.
|
||||||
|
|
|
||||||
|
|
@ -61,6 +61,8 @@ Use this file before making status claims. Prefer fresh VPS/GCP readbacks when c
|
||||||
- Lossless Helmer strict normalization: `docs/reports/leo-working-state-20260709/helmer-approve-claim-normalization-current.json`
|
- Lossless Helmer strict normalization: `docs/reports/leo-working-state-20260709/helmer-approve-claim-normalization-current.json`
|
||||||
- Approved claim-bundle lifecycle harness: `scripts/run_approve_claim_clone_canary.py`
|
- Approved claim-bundle lifecycle harness: `scripts/run_approve_claim_clone_canary.py`
|
||||||
- Approved claim-bundle isolated-container wrapper: `scripts/run_approve_claim_isolated_container_canary.sh`
|
- Approved claim-bundle isolated-container wrapper: `scripts/run_approve_claim_isolated_container_canary.sh`
|
||||||
|
- Live VPS source-composition clone checkpoint summary: `docs/reports/leo-working-state-20260709/leo-source-composition-clone-checkpoint-current.md`
|
||||||
|
- Live VPS source-composition clone checkpoint full JSON: `docs/reports/leo-working-state-20260709/leo-source-composition-clone-checkpoint-current.json`
|
||||||
- Claim/source preview: `docs/reports/leo-working-state-20260709/claim-source-contract-preview-current.md`
|
- Claim/source preview: `docs/reports/leo-working-state-20260709/claim-source-contract-preview-current.md`
|
||||||
- Mapped rich proposal plan: `docs/reports/leo-working-state-20260709/rich-proposal-creation-plan-mapped-current.md`
|
- Mapped rich proposal plan: `docs/reports/leo-working-state-20260709/rich-proposal-creation-plan-mapped-current.md`
|
||||||
- Mapped production apply packet, not executed in production: `docs/reports/leo-working-state-20260709/production-apply-packet-current.md`
|
- Mapped production apply packet, not executed in production: `docs/reports/leo-working-state-20260709/production-apply-packet-current.md`
|
||||||
|
|
@ -161,6 +163,7 @@ Use this file before making status claims. Prefer fresh VPS/GCP readbacks when c
|
||||||
- The highest-risk direct-claim cases are now green at handler level: `DC-02` recognizes approved Helmer staging row `a64df080-8502-42e2-98f4-9bbdecb8da73` as approved/unapplied, `DC-03` says the decision-matrix path is not shipped while matrix tables are absent, `DC-04` avoids a single-cause document pointer diagnosis, `DC-05` separates staging demo from authorized canonical apply, and `DC-06` separates `SOUL.md` runtime edits from canonical DB identity.
|
- The highest-risk direct-claim cases are now green at handler level: `DC-02` recognizes approved Helmer staging row `a64df080-8502-42e2-98f4-9bbdecb8da73` as approved/unapplied, `DC-03` says the decision-matrix path is not shipped while matrix tables are absent, `DC-04` avoids a single-cause document pointer diagnosis, `DC-05` separates staging demo from authorized canonical apply, and `DC-06` separates `SOUL.md` runtime edits from canonical DB identity.
|
||||||
- A strict canonical `add_edge` apply canary is live-proven.
|
- A strict canonical `add_edge` apply canary is live-proven.
|
||||||
- The repo-owned normal apply path now supports an atomic, insert-only `approve_claim` bundle behind separate `kb_review` and `kb_apply` credentials, a DB-role-to-reviewer principal mapping, and an immutable approval snapshot. Final source-bound isolated-container receipts pass `37/37` for both the generic bundle (`2` claims, `2` sources, `2` evidence rows, `1` edge, `1` reasoning tool) and the real losslessly normalized Helmer bundle (`5/13/17/7/1`). They prove exact payload-controlled projections and table deltas, reviewer/apply separation, denied direct proposal/evidence mutation, stale-payload refusal before canonical writes, replay refusal, source-hash rollback, exact agent/external-claim seeding, hardened role/function/ACL state, and independent inner/outer cleanup. PR #72 source later auto-synchronized to the VPS checkout and deploy stamp at merge commit `771679061f0a0b6463fb7d9cf530eb9d2d4e5440`; the timer journal says no Python changes and no service restart, the commit had no delta under the Leo profile source paths, the gateway stayed PID `2999690`/`NRestarts=0`, the apply worker stayed disabled/inactive, and read-only count endpoints remained `1837/4145/4670/4916/17/26`. This is a source checkout sync, not proof that the production permission migration ran, the worker was enabled, canonical rows changed, or byte-for-byte live content stayed equal. Helmer remains unapplied on production. See `pr72-vps-auto-deploy-runtime-nonchange-current.md`.
|
- The repo-owned normal apply path now supports an atomic, insert-only `approve_claim` bundle behind separate `kb_review` and `kb_apply` credentials, a DB-role-to-reviewer principal mapping, and an immutable approval snapshot. Final source-bound isolated-container receipts pass `37/37` for both the generic bundle (`2` claims, `2` sources, `2` evidence rows, `1` edge, `1` reasoning tool) and the real losslessly normalized Helmer bundle (`5/13/17/7/1`). They prove exact payload-controlled projections and table deltas, reviewer/apply separation, denied direct proposal/evidence mutation, stale-payload refusal before canonical writes, replay refusal, source-hash rollback, exact agent/external-claim seeding, hardened role/function/ACL state, and independent inner/outer cleanup. PR #72 source later auto-synchronized to the VPS checkout and deploy stamp at merge commit `771679061f0a0b6463fb7d9cf530eb9d2d4e5440`; the timer journal says no Python changes and no service restart, the commit had no delta under the Leo profile source paths, the gateway stayed PID `2999690`/`NRestarts=0`, the apply worker stayed disabled/inactive, and read-only count endpoints remained `1837/4145/4670/4916/17/26`. This is a source checkout sync, not proof that the production permission migration ran, the worker was enabled, canonical rows changed, or byte-for-byte live content stayed equal. Helmer remains unapplied on production. See `pr72-vps-auto-deploy-runtime-nonchange-current.md`.
|
||||||
|
- The live VPS source-composition checkpoint now passes `34/34` with zero errors on a fresh no-send full-data clone: Leo searched the current KB, extracted two atomic conflicting claims from a new hash-bound document and post, retained exact evidence/source links, staged a strict proposal, passed separated review and guarded apply, then reopened in a new handler process and discovered/reasoned over the exact canonical proposal, claims, source UUIDs, and conflict edge without supplied IDs while recalling the prior conversation marker. Production counts and row-content fingerprints, gate schema, gateway PID/restarts, and live bridge hashes stayed unchanged. All three experiment containers, anonymous volumes, private credentials, run directories, temporary profiles, and child processes were removed. This proves current-data clone composition and restarted reasoning, not Telegram-visible delivery, production apply, scheduled DB-to-identity recomposition, arbitrary-source breadth, or GCP parity. See `leo-source-composition-clone-checkpoint-current.md`.
|
||||||
- Helmer 7 Powers is full-coverage packet and ledger clone-proven, not production-applied.
|
- Helmer 7 Powers is full-coverage packet and ledger clone-proven, not production-applied.
|
||||||
- Mapped rich approved proposal packet is clone/rehearsal-proven for the supported subset, not production-applied.
|
- Mapped rich approved proposal packet is clone/rehearsal-proven for the supported subset, not production-applied.
|
||||||
- The strategy-anchor companion write for mapped rich proposal `14fa5ecc` is clone-proven and reversible, not production-applied.
|
- The strategy-anchor companion write for mapped rich proposal `14fa5ecc` is clone-proven and reversible, not production-applied.
|
||||||
|
|
|
||||||
File diff suppressed because it is too large
Load diff
|
|
@ -0,0 +1,68 @@
|
||||||
|
# Leo Source Composition Clone Checkpoint
|
||||||
|
|
||||||
|
## Result
|
||||||
|
|
||||||
|
- Status: `pass`, `34/34` checks, zero harness errors.
|
||||||
|
- Tier: live VPS, no-send, disposable full-data clone.
|
||||||
|
- Full receipt: `leo-source-composition-clone-checkpoint-current.json`.
|
||||||
|
- Receipt SHA-256: `897f2b48a5b85a32a069e154f246140421a16942acd85e32e4b8edc1d6738278`.
|
||||||
|
- Production behavior changed: no.
|
||||||
|
|
||||||
|
## What Ran
|
||||||
|
|
||||||
|
Leo received a previously unindexed operating document and a contradictory
|
||||||
|
post for `AURORA-A1CF3DF21D`. The model first searched the current clone KB,
|
||||||
|
then extracted two atomic claims, retained both supplied source-content hashes,
|
||||||
|
linked evidence from both sources, and represented the conflict bidirectionally.
|
||||||
|
Extraction changed no database rows.
|
||||||
|
|
||||||
|
The harness normalized the packet into strict proposal
|
||||||
|
`b84244ce-53c8-5172-8c5e-38814f4f4e95`, staged it as `pending_review`, performed
|
||||||
|
a separated operator review to `approved`, and ran guarded apply to `applied`.
|
||||||
|
All mutation occurred only in disposable clone `leo-compose-20260711-03`.
|
||||||
|
|
||||||
|
Exact clone deltas were:
|
||||||
|
|
||||||
|
- `kb_stage.kb_proposals`: `+1`
|
||||||
|
- `kb_stage.kb_proposal_approvals`: `+1`
|
||||||
|
- `public.claims`: `+2`
|
||||||
|
- `public.sources`: `+4`
|
||||||
|
- `public.claim_evidence`: `+4`
|
||||||
|
- `public.claim_edges`: `+2`
|
||||||
|
- `public.reasoning_tools`: `+0`
|
||||||
|
|
||||||
|
## Restarted Reasoning
|
||||||
|
|
||||||
|
A new isolated GatewayRunner child reused the same private persisted session,
|
||||||
|
recalled `composition-memory-live-20260711-03`, searched without generated IDs,
|
||||||
|
found and opened the applied proposal, read JSON evidence for both claims, read
|
||||||
|
the canonical contradiction edge, and returned every exact proposal, claim,
|
||||||
|
source, and edge ID in its final state receipt.
|
||||||
|
|
||||||
|
The transcript contained seven terminal calls. Every call had a result, none
|
||||||
|
was rejected or nonzero, all seven matched nonce-bound wrapper receipts, and the
|
||||||
|
two bounded/truncated transcript results reconciled to successful wrapper
|
||||||
|
receipts. The GatewayRunner exposed no delivery adapter or send tool.
|
||||||
|
|
||||||
|
## Production And Cleanup
|
||||||
|
|
||||||
|
Production remained byte-for-byte stable across the guarded fingerprints:
|
||||||
|
|
||||||
|
- gateway: PID `2999690`, `NRestarts=0`, start time unchanged;
|
||||||
|
- canonical counts: claims `1837`, sources `4145`, evidence `4670`, edges
|
||||||
|
`4916`, reasoning tools `17`, proposals `26`;
|
||||||
|
- every tracked production table MD5 unchanged;
|
||||||
|
- review/apply gate schema unchanged and still absent from production;
|
||||||
|
- live KB wrapper and bridge-skill hashes unchanged.
|
||||||
|
|
||||||
|
All three composition containers, their anonymous volumes, private credential
|
||||||
|
files, run directories, temporary profiles, and GatewayRunner children were
|
||||||
|
removed and independently verified absent.
|
||||||
|
|
||||||
|
## Claim Ceiling
|
||||||
|
|
||||||
|
This proves source extraction, hash-bound normalization, guarded canonical
|
||||||
|
composition, conflict-aware reasoning, and isolated handler/session survival on
|
||||||
|
a current full-data VPS clone. It does not prove Telegram-visible delivery,
|
||||||
|
production DB mutation, scheduled DB-to-identity recomposition, arbitrary-source
|
||||||
|
coverage beyond this case, or GCP parity.
|
||||||
209
scripts/bootstrap_clone_kb_gate.py
Normal file
209
scripts/bootstrap_clone_kb_gate.py
Normal file
|
|
@ -0,0 +1,209 @@
|
||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Install guarded review/apply prerequisites and ephemeral credentials in a disposable clone."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import secrets
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
HERE = Path(__file__).resolve().parent
|
||||||
|
sys.path.insert(0, str(HERE))
|
||||||
|
|
||||||
|
import apply_proposal as ap # noqa: E402
|
||||||
|
import run_leo_clone_bound_handler_checkpoint as bound # noqa: E402
|
||||||
|
|
||||||
|
|
||||||
|
def _psql(
|
||||||
|
container: str,
|
||||||
|
database: str,
|
||||||
|
sql: str,
|
||||||
|
*,
|
||||||
|
secret_values: tuple[str, ...] = (),
|
||||||
|
) -> None:
|
||||||
|
command = [
|
||||||
|
bound.SYSTEM_DOCKER,
|
||||||
|
"exec",
|
||||||
|
"-i",
|
||||||
|
container,
|
||||||
|
"psql",
|
||||||
|
"-U",
|
||||||
|
"postgres",
|
||||||
|
"-d",
|
||||||
|
database,
|
||||||
|
"-v",
|
||||||
|
"ON_ERROR_STOP=1",
|
||||||
|
"-At",
|
||||||
|
"-q",
|
||||||
|
]
|
||||||
|
result = subprocess.run(
|
||||||
|
command,
|
||||||
|
input=sql,
|
||||||
|
text=True,
|
||||||
|
capture_output=True,
|
||||||
|
env={"PATH": bound.SYSTEM_EXEC_PATH},
|
||||||
|
check=False,
|
||||||
|
)
|
||||||
|
if result.returncode != 0:
|
||||||
|
raise bound.CheckpointError(
|
||||||
|
f"clone gate SQL failed ({result.returncode}): "
|
||||||
|
+ bound.redact_text(result.stderr or result.stdout, secret_values=secret_values)
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _verify_login(container: str, database: str, role: str, password: str) -> None:
|
||||||
|
command = [
|
||||||
|
bound.SYSTEM_DOCKER,
|
||||||
|
"exec",
|
||||||
|
"-e",
|
||||||
|
"PGPASSWORD",
|
||||||
|
"-i",
|
||||||
|
container,
|
||||||
|
"psql",
|
||||||
|
"-U",
|
||||||
|
role,
|
||||||
|
"-h",
|
||||||
|
"127.0.0.1",
|
||||||
|
"-d",
|
||||||
|
database,
|
||||||
|
"-v",
|
||||||
|
"ON_ERROR_STOP=1",
|
||||||
|
"-At",
|
||||||
|
"-q",
|
||||||
|
"-c",
|
||||||
|
"select current_user;",
|
||||||
|
]
|
||||||
|
result = subprocess.run(
|
||||||
|
command,
|
||||||
|
text=True,
|
||||||
|
capture_output=True,
|
||||||
|
env={"PATH": bound.SYSTEM_EXEC_PATH, "PGPASSWORD": password},
|
||||||
|
check=False,
|
||||||
|
)
|
||||||
|
if result.returncode != 0 or result.stdout.strip() != role:
|
||||||
|
raise bound.CheckpointError(f"ephemeral {role} clone login verification failed")
|
||||||
|
|
||||||
|
|
||||||
|
def _write_secret(path: Path, key: str, value: str) -> None:
|
||||||
|
descriptor = os.open(
|
||||||
|
path,
|
||||||
|
os.O_WRONLY | os.O_CREAT | os.O_EXCL | getattr(os, "O_NOFOLLOW", 0),
|
||||||
|
0o600,
|
||||||
|
)
|
||||||
|
with os.fdopen(descriptor, "w", encoding="utf-8") as handle:
|
||||||
|
handle.write(f"{key}={value}\n")
|
||||||
|
handle.flush()
|
||||||
|
os.fsync(handle.fileno())
|
||||||
|
|
||||||
|
|
||||||
|
def bootstrap(container: str, database: str, output_dir: Path) -> dict[str, Any]:
|
||||||
|
target = bound.container_identity(container)
|
||||||
|
production = bound.container_identity(bound.PRODUCTION_CONTAINER)
|
||||||
|
isolation = bound.validate_disposable_target_identity(target, production)
|
||||||
|
target_id = str(target["id"])
|
||||||
|
output_dir = output_dir.resolve(strict=True)
|
||||||
|
stat = output_dir.stat()
|
||||||
|
if not output_dir.is_dir() or stat.st_uid != os.geteuid() or stat.st_mode & 0o077:
|
||||||
|
raise bound.CheckpointError("--output-dir must be an existing private directory owned by the current user")
|
||||||
|
review_file = output_dir / "kb-review.env"
|
||||||
|
apply_file = output_dir / "kb-apply.env"
|
||||||
|
if review_file.exists() or review_file.is_symlink() or apply_file.exists() or apply_file.is_symlink():
|
||||||
|
raise bound.CheckpointError("clone credential files already exist; use a fresh private run directory")
|
||||||
|
review_password = secrets.token_urlsafe(36)
|
||||||
|
apply_password = secrets.token_urlsafe(36)
|
||||||
|
prereqs = (HERE / "kb_apply_prereqs.sql").read_text(encoding="utf-8")
|
||||||
|
_psql(
|
||||||
|
target_id,
|
||||||
|
database,
|
||||||
|
"""do $bootstrap$
|
||||||
|
begin
|
||||||
|
if not exists (select 1 from pg_roles where rolname = 'kb_apply') then
|
||||||
|
create role kb_apply login;
|
||||||
|
end if;
|
||||||
|
end
|
||||||
|
$bootstrap$;
|
||||||
|
""",
|
||||||
|
)
|
||||||
|
_psql(target_id, database, prereqs)
|
||||||
|
_psql(
|
||||||
|
target_id,
|
||||||
|
database,
|
||||||
|
"alter role kb_review password "
|
||||||
|
+ ap.sql_literal(review_password)
|
||||||
|
+ ";\nalter role kb_apply password "
|
||||||
|
+ ap.sql_literal(apply_password)
|
||||||
|
+ ";\n",
|
||||||
|
secret_values=(review_password, apply_password),
|
||||||
|
)
|
||||||
|
_verify_login(target_id, database, "kb_review", review_password)
|
||||||
|
_verify_login(target_id, database, "kb_apply", apply_password)
|
||||||
|
manifest = bound._psql_json(
|
||||||
|
target_id,
|
||||||
|
database,
|
||||||
|
"""\\set ON_ERROR_STOP on
|
||||||
|
begin transaction read only;
|
||||||
|
select jsonb_build_object(
|
||||||
|
'approval_table', to_regclass('kb_stage.kb_proposal_approvals')::text,
|
||||||
|
'review_principals_table', to_regclass('kb_stage.kb_review_principals')::text,
|
||||||
|
'review_role_exists', exists(select 1 from pg_roles where rolname = 'kb_review'),
|
||||||
|
'apply_role_exists', exists(select 1 from pg_roles where rolname = 'kb_apply')
|
||||||
|
)::text;
|
||||||
|
rollback;
|
||||||
|
""",
|
||||||
|
)
|
||||||
|
if manifest != {
|
||||||
|
"approval_table": "kb_stage.kb_proposal_approvals",
|
||||||
|
"review_principals_table": "kb_stage.kb_review_principals",
|
||||||
|
"review_role_exists": True,
|
||||||
|
"apply_role_exists": True,
|
||||||
|
}:
|
||||||
|
raise bound.CheckpointError("clone gate manifest is incomplete: " + json.dumps(manifest, sort_keys=True))
|
||||||
|
_write_secret(review_file, "KB_REVIEW_PASSWORD", review_password)
|
||||||
|
try:
|
||||||
|
_write_secret(apply_file, "KB_APPLY_DB_PASSWORD", apply_password)
|
||||||
|
except BaseException:
|
||||||
|
review_file.unlink(missing_ok=True)
|
||||||
|
raise
|
||||||
|
review_password = ""
|
||||||
|
apply_password = ""
|
||||||
|
return {
|
||||||
|
"status": "ready",
|
||||||
|
"bound_container_id": target_id,
|
||||||
|
"database": database,
|
||||||
|
"isolation_checks": isolation,
|
||||||
|
"gate_manifest": manifest,
|
||||||
|
"review_secret_file": str(review_file),
|
||||||
|
"apply_secret_file": str(apply_file),
|
||||||
|
"secret_values_or_hashes_recorded": False,
|
||||||
|
"login_roles_verified": ["kb_review", "kb_apply"],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def parse_args(argv: list[str] | None = None) -> argparse.Namespace:
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
|
parser.add_argument("--container", required=True)
|
||||||
|
parser.add_argument("--db", required=True)
|
||||||
|
parser.add_argument("--output-dir", required=True, type=Path)
|
||||||
|
return parser.parse_args(argv)
|
||||||
|
|
||||||
|
|
||||||
|
def main(argv: list[str] | None = None) -> int:
|
||||||
|
args = parse_args(argv)
|
||||||
|
try:
|
||||||
|
if args.container == bound.PRODUCTION_CONTAINER:
|
||||||
|
raise bound.CheckpointError("--container must be a disposable non-production target")
|
||||||
|
result = bootstrap(args.container, args.db, args.output_dir)
|
||||||
|
except (OSError, bound.CheckpointError) as exc:
|
||||||
|
print(json.dumps({"status": "rejected", "error": str(exc)}, sort_keys=True))
|
||||||
|
return 2
|
||||||
|
print(json.dumps(result, sort_keys=True))
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
|
|
@ -33,9 +33,8 @@ import apply_proposal as ap # noqa: E402
|
||||||
import kb_proposal_review_packet as review_packet # noqa: E402
|
import kb_proposal_review_packet as review_packet # noqa: E402
|
||||||
import kb_rich_proposal_creation_plan as rich_plan # noqa: E402
|
import kb_rich_proposal_creation_plan as rich_plan # noqa: E402
|
||||||
|
|
||||||
UUID_RE = re.compile(
|
UUID_RE = re.compile(r"^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$")
|
||||||
r"^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$"
|
SHA256_RE = re.compile(r"^[0-9a-fA-F]{64}$")
|
||||||
)
|
|
||||||
RICH_CANDIDATE_COLLECTIONS = (
|
RICH_CANDIDATE_COLLECTIONS = (
|
||||||
("claim_candidates", "claim_key", "claim"),
|
("claim_candidates", "claim_key", "claim"),
|
||||||
("proposed_claims", "claim_key", "claim"),
|
("proposed_claims", "claim_key", "claim"),
|
||||||
|
|
@ -154,6 +153,21 @@ def _candidate_inventory(payload: dict[str, Any]) -> dict[str, Any]:
|
||||||
inventory["blocked_fragments"].append(block)
|
inventory["blocked_fragments"].append(block)
|
||||||
continue
|
continue
|
||||||
|
|
||||||
|
if kind == "source":
|
||||||
|
supplied_hash = candidate.get("content_sha256") or candidate.get("hash")
|
||||||
|
if supplied_hash is not None and (
|
||||||
|
not isinstance(supplied_hash, str) or not SHA256_RE.fullmatch(supplied_hash)
|
||||||
|
):
|
||||||
|
block = _blocked(
|
||||||
|
"source_candidate",
|
||||||
|
f"{location} has an invalid content_sha256/hash; expected 64 hexadecimal characters",
|
||||||
|
["content_sha256"],
|
||||||
|
candidate,
|
||||||
|
)
|
||||||
|
block["location"] = location
|
||||||
|
inventory["blocked_fragments"].append(block)
|
||||||
|
continue
|
||||||
|
|
||||||
inventory[f"{kind}_items"].append(
|
inventory[f"{kind}_items"].append(
|
||||||
{
|
{
|
||||||
"collection": collection,
|
"collection": collection,
|
||||||
|
|
@ -183,9 +197,7 @@ def _candidate_inventory(payload: dict[str, Any]) -> dict[str, Any]:
|
||||||
return inventory
|
return inventory
|
||||||
|
|
||||||
|
|
||||||
def _planned_row_accounting(
|
def _planned_row_accounting(plan: dict[str, Any], inventory: dict[str, Any]) -> tuple[dict[str, Any], list[str]]:
|
||||||
plan: dict[str, Any], inventory: dict[str, Any]
|
|
||||||
) -> tuple[dict[str, Any], list[str]]:
|
|
||||||
"""Reconcile every validated input candidate to exactly one planned row."""
|
"""Reconcile every validated input candidate to exactly one planned row."""
|
||||||
rows = plan.get("planned_rows") if isinstance(plan.get("planned_rows"), dict) else {}
|
rows = plan.get("planned_rows") if isinstance(plan.get("planned_rows"), dict) else {}
|
||||||
generated = plan.get("generated_ids") if isinstance(plan.get("generated_ids"), dict) else {}
|
generated = plan.get("generated_ids") if isinstance(plan.get("generated_ids"), dict) else {}
|
||||||
|
|
@ -264,9 +276,7 @@ def _normalization_manifest(plan: dict[str, Any], accounting: dict[str, Any]) ->
|
||||||
for row in rows["claims"]:
|
for row in rows["claims"]:
|
||||||
mapping = row.get("type_normalization")
|
mapping = row.get("type_normalization")
|
||||||
if mapping:
|
if mapping:
|
||||||
claim_type_mappings.append(
|
claim_type_mappings.append({"claim_key": row.get("claim_key"), "claim_id": row.get("id"), **dict(mapping)})
|
||||||
{"claim_key": row.get("claim_key"), "claim_id": row.get("id"), **dict(mapping)}
|
|
||||||
)
|
|
||||||
for row in rows["sources"]:
|
for row in rows["sources"]:
|
||||||
mapping = row.get("source_type_normalization")
|
mapping = row.get("source_type_normalization")
|
||||||
if mapping:
|
if mapping:
|
||||||
|
|
@ -345,17 +355,11 @@ def _strict_approve_claim_child(
|
||||||
"source_proposal_id": proposal.get("id"),
|
"source_proposal_id": proposal.get("id"),
|
||||||
"agent_id": agent_id,
|
"agent_id": agent_id,
|
||||||
"claims": [
|
"claims": [
|
||||||
{
|
{key: row.get(key) for key in ("id", "type", "text", "status", "confidence", "tags", "created_by")}
|
||||||
key: row.get(key)
|
|
||||||
for key in ("id", "type", "text", "status", "confidence", "tags", "created_by")
|
|
||||||
}
|
|
||||||
for row in rows["claims"]
|
for row in rows["claims"]
|
||||||
],
|
],
|
||||||
"sources": [
|
"sources": [
|
||||||
{
|
{key: row.get(key) for key in ("id", "source_type", "url", "storage_path", "excerpt", "hash", "created_by")}
|
||||||
key: row.get(key)
|
|
||||||
for key in ("id", "source_type", "url", "storage_path", "excerpt", "hash", "created_by")
|
|
||||||
}
|
|
||||||
for row in rows["sources"]
|
for row in rows["sources"]
|
||||||
],
|
],
|
||||||
"evidence": [
|
"evidence": [
|
||||||
|
|
@ -379,10 +383,7 @@ def _strict_approve_claim_child(
|
||||||
for row in rows["claim_edges"]
|
for row in rows["claim_edges"]
|
||||||
],
|
],
|
||||||
"reasoning_tools": [
|
"reasoning_tools": [
|
||||||
{
|
{key: row.get(key) for key in ("id", "agent_id", "name", "description", "category")}
|
||||||
key: row.get(key)
|
|
||||||
for key in ("id", "agent_id", "name", "description", "category")
|
|
||||||
}
|
|
||||||
for row in rows["reasoning_tools"]
|
for row in rows["reasoning_tools"]
|
||||||
],
|
],
|
||||||
"normalization_manifest": normalization_manifest,
|
"normalization_manifest": normalization_manifest,
|
||||||
|
|
@ -497,13 +498,17 @@ def _normalize_evidence(payload: dict[str, Any]) -> tuple[list[dict[str, Any]],
|
||||||
missing.append("canonical claim_id")
|
missing.append("canonical claim_id")
|
||||||
if not _first_uuid(fragment, ("source_id", "source_uuid", "canonical_source_id")):
|
if not _first_uuid(fragment, ("source_id", "source_uuid", "canonical_source_id")):
|
||||||
missing.append("canonical source_id")
|
missing.append("canonical source_id")
|
||||||
blocked.append(_blocked("evidence", "evidence fragment is not a strict attach_evidence contract", missing, fragment))
|
blocked.append(
|
||||||
|
_blocked("evidence", "evidence fragment is not a strict attach_evidence contract", missing, fragment)
|
||||||
|
)
|
||||||
|
|
||||||
strict = [_strict_child("attach_evidence", {"evidence": evidence_rows}, evidence_rows)] if evidence_rows else []
|
strict = [_strict_child("attach_evidence", {"evidence": evidence_rows}, evidence_rows)] if evidence_rows else []
|
||||||
return strict, blocked
|
return strict, blocked
|
||||||
|
|
||||||
|
|
||||||
def _normalize_strategy(proposal: dict[str, Any], payload: dict[str, Any]) -> tuple[list[dict[str, Any]], list[dict[str, Any]]]:
|
def _normalize_strategy(
|
||||||
|
proposal: dict[str, Any], payload: dict[str, Any]
|
||||||
|
) -> tuple[list[dict[str, Any]], list[dict[str, Any]]]:
|
||||||
if proposal.get("proposal_type") != "revise_strategy":
|
if proposal.get("proposal_type") != "revise_strategy":
|
||||||
return [], []
|
return [], []
|
||||||
strategy_payload = payload.get("apply_payload") or {
|
strategy_payload = payload.get("apply_payload") or {
|
||||||
|
|
@ -534,9 +539,7 @@ def _dedupe_children(children: list[dict[str, Any]]) -> list[dict[str, Any]]:
|
||||||
return out
|
return out
|
||||||
|
|
||||||
|
|
||||||
def normalize_proposal(
|
def normalize_proposal(proposal: dict[str, Any], mappings: dict[str, Any] | None = None) -> dict[str, Any]:
|
||||||
proposal: dict[str, Any], mappings: dict[str, Any] | None = None
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
payload = proposal.get("payload") or {}
|
payload = proposal.get("payload") or {}
|
||||||
if not isinstance(payload, dict):
|
if not isinstance(payload, dict):
|
||||||
payload = {}
|
payload = {}
|
||||||
|
|
|
||||||
|
|
@ -134,7 +134,14 @@ def _source_excerpt(source: dict[str, Any]) -> str:
|
||||||
|
|
||||||
metadata = {
|
metadata = {
|
||||||
key: source.get(key)
|
key: source.get(key)
|
||||||
for key in ("title", "source_key", "source_quality", "usage_limits", "claim_id")
|
for key in (
|
||||||
|
"title",
|
||||||
|
"source_key",
|
||||||
|
"source_quality",
|
||||||
|
"usage_limits",
|
||||||
|
"claim_id",
|
||||||
|
"content_sha256",
|
||||||
|
)
|
||||||
if source.get(key) is not None
|
if source.get(key) is not None
|
||||||
}
|
}
|
||||||
if metadata:
|
if metadata:
|
||||||
|
|
@ -157,7 +164,11 @@ def _normalize_source_type(raw_type: Any) -> tuple[str, dict[str, Any] | None]:
|
||||||
if raw_text in LIVE_SOURCE_TYPES:
|
if raw_text in LIVE_SOURCE_TYPES:
|
||||||
return raw_text, None
|
return raw_text, None
|
||||||
mapped = SOURCE_TYPE_ALIASES.get(raw_text, "other")
|
mapped = SOURCE_TYPE_ALIASES.get(raw_text, "other")
|
||||||
return mapped, {"from": raw_text, "to": mapped, "reason": "proposal source_type collapsed to live sources.source_type"}
|
return mapped, {
|
||||||
|
"from": raw_text,
|
||||||
|
"to": mapped,
|
||||||
|
"reason": "proposal source_type collapsed to live sources.source_type",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
def _claim_text(candidate: dict[str, Any]) -> str:
|
def _claim_text(candidate: dict[str, Any]) -> str:
|
||||||
|
|
@ -177,8 +188,16 @@ def _normalize_claim_type(raw_type: Any) -> tuple[str, dict[str, Any] | None]:
|
||||||
return part, {"from": raw_text, "to": part, "reason": "proposal subtype collapsed to live claims.type"}
|
return part, {"from": raw_text, "to": part, "reason": "proposal subtype collapsed to live claims.type"}
|
||||||
if part in CLAIM_TYPE_ALIASES:
|
if part in CLAIM_TYPE_ALIASES:
|
||||||
mapped = CLAIM_TYPE_ALIASES[part]
|
mapped = CLAIM_TYPE_ALIASES[part]
|
||||||
return mapped, {"from": raw_text, "to": mapped, "reason": "proposal governance/process subtype mapped to meta"}
|
return mapped, {
|
||||||
return "meta", {"from": raw_text, "to": "meta", "reason": "unsupported proposal type requires review; meta used for rehearsal"}
|
"from": raw_text,
|
||||||
|
"to": mapped,
|
||||||
|
"reason": "proposal governance/process subtype mapped to meta",
|
||||||
|
}
|
||||||
|
return "meta", {
|
||||||
|
"from": raw_text,
|
||||||
|
"to": "meta",
|
||||||
|
"reason": "unsupported proposal type requires review; meta used for rehearsal",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
def _claim_body_excerpt(proposal: dict[str, Any], candidate: dict[str, Any]) -> str:
|
def _claim_body_excerpt(proposal: dict[str, Any], candidate: dict[str, Any]) -> str:
|
||||||
|
|
@ -200,7 +219,9 @@ def _claim_body_excerpt(proposal: dict[str, Any], candidate: dict[str, Any]) ->
|
||||||
return _compact_text(json.dumps(body_packet, ensure_ascii=False, sort_keys=True))
|
return _compact_text(json.dumps(body_packet, ensure_ascii=False, sort_keys=True))
|
||||||
|
|
||||||
|
|
||||||
def _candidate_claim_rows(proposal: dict[str, Any], payload: dict[str, Any]) -> tuple[list[dict[str, Any]], dict[str, str]]:
|
def _candidate_claim_rows(
|
||||||
|
proposal: dict[str, Any], payload: dict[str, Any]
|
||||||
|
) -> tuple[list[dict[str, Any]], dict[str, str]]:
|
||||||
rows: list[dict[str, Any]] = []
|
rows: list[dict[str, Any]] = []
|
||||||
key_to_id: dict[str, str] = {}
|
key_to_id: dict[str, str] = {}
|
||||||
proposal_id = str(proposal.get("id") or "")
|
proposal_id = str(proposal.get("id") or "")
|
||||||
|
|
@ -250,6 +271,7 @@ def _source_rows(
|
||||||
excerpt = _source_excerpt(source)
|
excerpt = _source_excerpt(source)
|
||||||
raw_source_type = _source_type(source)
|
raw_source_type = _source_type(source)
|
||||||
source_type, source_type_normalization = _normalize_source_type(raw_source_type)
|
source_type, source_type_normalization = _normalize_source_type(raw_source_type)
|
||||||
|
supplied_hash = source.get("content_sha256") or source.get("hash")
|
||||||
row = {
|
row = {
|
||||||
"id": source_id,
|
"id": source_id,
|
||||||
"source_key": source_key,
|
"source_key": source_key,
|
||||||
|
|
@ -259,12 +281,16 @@ def _source_rows(
|
||||||
"url": source.get("url"),
|
"url": source.get("url"),
|
||||||
"storage_path": source.get("local_text") or source.get("local_snapshot") or source.get("storage_path"),
|
"storage_path": source.get("local_text") or source.get("local_snapshot") or source.get("storage_path"),
|
||||||
"excerpt": excerpt,
|
"excerpt": excerpt,
|
||||||
"hash": _stable_hash({"proposal_id": proposal_id, "source_key": source_key, "excerpt": excerpt}),
|
"hash": str(supplied_hash).lower()
|
||||||
|
if supplied_hash
|
||||||
|
else _stable_hash({"proposal_id": proposal_id, "source_key": source_key, "excerpt": excerpt}),
|
||||||
"created_by": created_by,
|
"created_by": created_by,
|
||||||
}
|
}
|
||||||
rows.append(row)
|
rows.append(row)
|
||||||
|
|
||||||
claim_candidates = {str(candidate.get("claim_key") or ""): candidate for candidate in _claim_candidate_items(payload)}
|
claim_candidates = {
|
||||||
|
str(candidate.get("claim_key") or ""): candidate for candidate in _claim_candidate_items(payload)
|
||||||
|
}
|
||||||
for claim in claim_rows:
|
for claim in claim_rows:
|
||||||
claim_key = claim["claim_key"]
|
claim_key = claim["claim_key"]
|
||||||
candidate = claim_candidates.get(claim_key)
|
candidate = claim_candidates.get(claim_key)
|
||||||
|
|
@ -440,7 +466,15 @@ def _plan_evidence(
|
||||||
key = (claim_id, source_id, role)
|
key = (claim_id, source_id, role)
|
||||||
if key not in seen:
|
if key not in seen:
|
||||||
seen.add(key)
|
seen.add(key)
|
||||||
planned.append({"claim_id": claim_id, "source_id": source_id, "role": role, "weight": None, "source_fragment": fragment})
|
planned.append(
|
||||||
|
{
|
||||||
|
"claim_id": claim_id,
|
||||||
|
"source_id": source_id,
|
||||||
|
"role": role,
|
||||||
|
"weight": None,
|
||||||
|
"source_fragment": fragment,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
for candidate in _claim_candidate_items(payload):
|
for candidate in _claim_candidate_items(payload):
|
||||||
claim_key = str(candidate.get("claim_key") or "")
|
claim_key = str(candidate.get("claim_key") or "")
|
||||||
|
|
@ -563,47 +597,47 @@ def _reasoning_tool_rows(proposal: dict[str, Any], payload: dict[str, Any]) -> l
|
||||||
|
|
||||||
def _reasoning_tool_insert_sql(row: dict[str, Any]) -> str:
|
def _reasoning_tool_insert_sql(row: dict[str, Any]) -> str:
|
||||||
return f"""insert into public.reasoning_tools (id, agent_id, name, description, category)
|
return f"""insert into public.reasoning_tools (id, agent_id, name, description, category)
|
||||||
select {_sql_literal(row['id'])}::uuid, {_sql_literal(row['agent_id'])}::uuid,
|
select {_sql_literal(row["id"])}::uuid, {_sql_literal(row["agent_id"])}::uuid,
|
||||||
{_sql_literal(row['name'])}, {_sql_literal(row['description'])}, {_sql_literal(row['category'])}
|
{_sql_literal(row["name"])}, {_sql_literal(row["description"])}, {_sql_literal(row["category"])}
|
||||||
where not exists (select 1 from public.reasoning_tools where id = {_sql_literal(row['id'])}::uuid);"""
|
where not exists (select 1 from public.reasoning_tools where id = {_sql_literal(row["id"])}::uuid);"""
|
||||||
|
|
||||||
|
|
||||||
def _claim_insert_sql(row: dict[str, Any]) -> str:
|
def _claim_insert_sql(row: dict[str, Any]) -> str:
|
||||||
return f"""insert into public.claims (id, type, text, status, confidence, tags, created_by)
|
return f"""insert into public.claims (id, type, text, status, confidence, tags, created_by)
|
||||||
select {_sql_literal(row['id'])}::uuid, {_sql_literal(row['type'])}, {_sql_literal(row['text'])},
|
select {_sql_literal(row["id"])}::uuid, {_sql_literal(row["type"])}, {_sql_literal(row["text"])},
|
||||||
{_sql_literal(row['status'])}, {_sql_literal(row['confidence'])}, {_sql_text_array(row['tags'])},
|
{_sql_literal(row["status"])}, {_sql_literal(row["confidence"])}, {_sql_text_array(row["tags"])},
|
||||||
{_sql_literal(row['created_by'])}::uuid
|
{_sql_literal(row["created_by"])}::uuid
|
||||||
where not exists (select 1 from public.claims where id = {_sql_literal(row['id'])}::uuid);"""
|
where not exists (select 1 from public.claims where id = {_sql_literal(row["id"])}::uuid);"""
|
||||||
|
|
||||||
|
|
||||||
def _source_insert_sql(row: dict[str, Any]) -> str:
|
def _source_insert_sql(row: dict[str, Any]) -> str:
|
||||||
return f"""insert into public.sources (id, source_type, url, storage_path, excerpt, hash, created_by)
|
return f"""insert into public.sources (id, source_type, url, storage_path, excerpt, hash, created_by)
|
||||||
select {_sql_literal(row['id'])}::uuid, {_sql_literal(row['source_type'])}, {_sql_literal(row['url'])},
|
select {_sql_literal(row["id"])}::uuid, {_sql_literal(row["source_type"])}, {_sql_literal(row["url"])},
|
||||||
{_sql_literal(row['storage_path'])}, {_sql_literal(row['excerpt'])}, {_sql_literal(row['hash'])},
|
{_sql_literal(row["storage_path"])}, {_sql_literal(row["excerpt"])}, {_sql_literal(row["hash"])},
|
||||||
{_sql_literal(row['created_by'])}::uuid
|
{_sql_literal(row["created_by"])}::uuid
|
||||||
where not exists (select 1 from public.sources where id = {_sql_literal(row['id'])}::uuid);"""
|
where not exists (select 1 from public.sources where id = {_sql_literal(row["id"])}::uuid);"""
|
||||||
|
|
||||||
|
|
||||||
def _edge_insert_sql(row: dict[str, Any]) -> str:
|
def _edge_insert_sql(row: dict[str, Any]) -> str:
|
||||||
return f"""insert into public.claim_edges (from_claim, to_claim, edge_type, weight)
|
return f"""insert into public.claim_edges (from_claim, to_claim, edge_type, weight)
|
||||||
select {_sql_literal(row['from_claim'])}::uuid, {_sql_literal(row['to_claim'])}::uuid,
|
select {_sql_literal(row["from_claim"])}::uuid, {_sql_literal(row["to_claim"])}::uuid,
|
||||||
{_sql_literal(row['edge_type'])}::edge_type, {_sql_literal(row['weight'])}
|
{_sql_literal(row["edge_type"])}::edge_type, {_sql_literal(row["weight"])}
|
||||||
where not exists (
|
where not exists (
|
||||||
select 1 from public.claim_edges
|
select 1 from public.claim_edges
|
||||||
where from_claim = {_sql_literal(row['from_claim'])}::uuid
|
where from_claim = {_sql_literal(row["from_claim"])}::uuid
|
||||||
and to_claim = {_sql_literal(row['to_claim'])}::uuid
|
and to_claim = {_sql_literal(row["to_claim"])}::uuid
|
||||||
and edge_type = {_sql_literal(row['edge_type'])}::edge_type);"""
|
and edge_type = {_sql_literal(row["edge_type"])}::edge_type);"""
|
||||||
|
|
||||||
|
|
||||||
def _evidence_insert_sql(row: dict[str, Any]) -> str:
|
def _evidence_insert_sql(row: dict[str, Any]) -> str:
|
||||||
return f"""insert into public.claim_evidence (claim_id, source_id, role, weight)
|
return f"""insert into public.claim_evidence (claim_id, source_id, role, weight)
|
||||||
select {_sql_literal(row['claim_id'])}::uuid, {_sql_literal(row['source_id'])}::uuid,
|
select {_sql_literal(row["claim_id"])}::uuid, {_sql_literal(row["source_id"])}::uuid,
|
||||||
{_sql_literal(row['role'])}::evidence_role, {_sql_literal(row['weight'])}
|
{_sql_literal(row["role"])}::evidence_role, {_sql_literal(row["weight"])}
|
||||||
where not exists (
|
where not exists (
|
||||||
select 1 from public.claim_evidence
|
select 1 from public.claim_evidence
|
||||||
where claim_id = {_sql_literal(row['claim_id'])}::uuid
|
where claim_id = {_sql_literal(row["claim_id"])}::uuid
|
||||||
and source_id = {_sql_literal(row['source_id'])}::uuid
|
and source_id = {_sql_literal(row["source_id"])}::uuid
|
||||||
and role = {_sql_literal(row['role'])}::evidence_role);"""
|
and role = {_sql_literal(row["role"])}::evidence_role);"""
|
||||||
|
|
||||||
|
|
||||||
def build_rollback_sql(plan: dict[str, Any]) -> str:
|
def build_rollback_sql(plan: dict[str, Any]) -> str:
|
||||||
|
|
@ -627,7 +661,9 @@ def build_creation_plan(proposal: dict[str, Any], mappings: dict[str, Any] | Non
|
||||||
claim_rows, claim_key_to_id = _candidate_claim_rows(proposal, payload)
|
claim_rows, claim_key_to_id = _candidate_claim_rows(proposal, payload)
|
||||||
source_rows, source_key_to_id, body_source_by_claim_key = _source_rows(proposal, payload, claim_rows)
|
source_rows, source_key_to_id, body_source_by_claim_key = _source_rows(proposal, payload, claim_rows)
|
||||||
edge_rows, blocked_edges = _plan_edges(payload, claim_key_to_id, mappings)
|
edge_rows, blocked_edges = _plan_edges(payload, claim_key_to_id, mappings)
|
||||||
evidence_rows, blocked_evidence = _plan_evidence(payload, claim_key_to_id, source_key_to_id, body_source_by_claim_key)
|
evidence_rows, blocked_evidence = _plan_evidence(
|
||||||
|
payload, claim_key_to_id, source_key_to_id, body_source_by_claim_key
|
||||||
|
)
|
||||||
reasoning_tool_rows = _reasoning_tool_rows(proposal, payload)
|
reasoning_tool_rows = _reasoning_tool_rows(proposal, payload)
|
||||||
|
|
||||||
old_claim = _payload_dict(payload, "old_claim")
|
old_claim = _payload_dict(payload, "old_claim")
|
||||||
|
|
|
||||||
2374
scripts/run_leo_clone_bound_handler_checkpoint.py
Normal file
2374
scripts/run_leo_clone_bound_handler_checkpoint.py
Normal file
File diff suppressed because it is too large
Load diff
1093
scripts/run_leo_clone_composition_checkpoint.py
Normal file
1093
scripts/run_leo_clone_composition_checkpoint.py
Normal file
File diff suppressed because it is too large
Load diff
1583
scripts/run_leo_clone_lifecycle_checkpoint.py
Normal file
1583
scripts/run_leo_clone_lifecycle_checkpoint.py
Normal file
File diff suppressed because it is too large
Load diff
221
scripts/stage_normalized_proposal.py
Normal file
221
scripts/stage_normalized_proposal.py
Normal file
|
|
@ -0,0 +1,221 @@
|
||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Normalize one rich KB packet and stage its strict child in a disposable clone."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
import uuid
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
HERE = Path(__file__).resolve().parent
|
||||||
|
sys.path.insert(0, str(HERE))
|
||||||
|
|
||||||
|
import apply_proposal as ap # noqa: E402
|
||||||
|
import kb_proposal_normalize as normalize # noqa: E402
|
||||||
|
import run_leo_clone_bound_handler_checkpoint as bound # noqa: E402
|
||||||
|
|
||||||
|
|
||||||
|
def canonical_sha256(value: Any) -> str:
|
||||||
|
encoded = json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=True).encode()
|
||||||
|
return hashlib.sha256(encoded).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def _uuid(value: Any, label: str) -> str:
|
||||||
|
try:
|
||||||
|
return str(uuid.UUID(str(value)))
|
||||||
|
except ValueError as exc:
|
||||||
|
raise bound.CheckpointError(f"{label} must be a full UUID") from exc
|
||||||
|
|
||||||
|
|
||||||
|
def prepare_normalized_child(parent: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
parent_id = _uuid(parent.get("id"), "parent proposal id")
|
||||||
|
preview = normalize.normalize_proposal(parent)
|
||||||
|
if preview.get("normalization_state") != "strict_children_ready" or preview.get("strict_child_count") != 1:
|
||||||
|
raise bound.CheckpointError(
|
||||||
|
"rich proposal did not normalize to exactly one strict child: "
|
||||||
|
+ json.dumps(
|
||||||
|
{
|
||||||
|
"normalization_state": preview.get("normalization_state"),
|
||||||
|
"strict_child_count": preview.get("strict_child_count"),
|
||||||
|
"blocked_fragments": preview.get("blocked_fragments"),
|
||||||
|
},
|
||||||
|
sort_keys=True,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
strict = preview["strict_child_proposals"][0]
|
||||||
|
proposal_type = strict.get("proposal_type")
|
||||||
|
if proposal_type not in ap.APPLYABLE_TYPES:
|
||||||
|
raise bound.CheckpointError(f"normalized child type is not applyable: {proposal_type!r}")
|
||||||
|
payload = json.loads(json.dumps(strict.get("payload"), sort_keys=True))
|
||||||
|
apply_payload = payload.get("apply_payload") if isinstance(payload, dict) else None
|
||||||
|
if not isinstance(apply_payload, dict):
|
||||||
|
raise bound.CheckpointError("normalized child has no strict payload.apply_payload object")
|
||||||
|
manifest = apply_payload.setdefault("normalization_manifest", {})
|
||||||
|
if not isinstance(manifest, dict):
|
||||||
|
raise bound.CheckpointError("normalized child normalization_manifest must be an object")
|
||||||
|
parent_packet_sha256 = canonical_sha256(parent)
|
||||||
|
manifest["parent_packet_sha256"] = parent_packet_sha256
|
||||||
|
child_payload_sha256 = canonical_sha256(payload)
|
||||||
|
child_id = str(
|
||||||
|
uuid.uuid5(
|
||||||
|
uuid.NAMESPACE_URL,
|
||||||
|
f"livingip:normalized-strict-child:{parent_id}:{proposal_type}:{child_payload_sha256}",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
source_ref = f"normalized:{parent_id}:{child_payload_sha256[:16]}"
|
||||||
|
child = {
|
||||||
|
"id": child_id,
|
||||||
|
"proposal_type": proposal_type,
|
||||||
|
"status": "pending_review",
|
||||||
|
"proposed_by_handle": parent.get("proposed_by_handle"),
|
||||||
|
"proposed_by_agent_id": parent.get("proposed_by_agent_id"),
|
||||||
|
"channel": parent.get("channel") or "clone_composition",
|
||||||
|
"source_ref": source_ref,
|
||||||
|
"rationale": f"Strict normalized child of rich source packet {parent_id}.",
|
||||||
|
"payload": payload,
|
||||||
|
"parent_packet_sha256": parent_packet_sha256,
|
||||||
|
"payload_sha256": child_payload_sha256,
|
||||||
|
}
|
||||||
|
validation_snapshot = {
|
||||||
|
**child,
|
||||||
|
"status": "approved",
|
||||||
|
"reviewed_by_handle": "normalization-preflight",
|
||||||
|
"reviewed_by_agent_id": None,
|
||||||
|
"reviewed_at": "2000-01-01T00:00:00+00:00",
|
||||||
|
"review_note": "Build-only validation of the exact normalized payload.",
|
||||||
|
}
|
||||||
|
ap.build_apply_sql(validation_snapshot, ap.SERVICE_AGENT_HANDLE)
|
||||||
|
return child
|
||||||
|
|
||||||
|
|
||||||
|
def build_stage_sql(child: dict[str, Any]) -> str:
|
||||||
|
proposal_id = ap.sql_literal(_uuid(child.get("id"), "child proposal id"))
|
||||||
|
proposal_type = ap.sql_literal(str(child["proposal_type"]))
|
||||||
|
proposed_by_handle = ap.sql_literal(str(child.get("proposed_by_handle") or ""))
|
||||||
|
proposed_by_agent_id = child.get("proposed_by_agent_id")
|
||||||
|
agent_sql = (
|
||||||
|
f"{ap.sql_literal(_uuid(proposed_by_agent_id, 'proposed_by_agent_id'))}::uuid"
|
||||||
|
if proposed_by_agent_id
|
||||||
|
else "null::uuid"
|
||||||
|
)
|
||||||
|
channel = ap.sql_literal(str(child["channel"]))
|
||||||
|
source_ref = ap.sql_literal(str(child["source_ref"]))
|
||||||
|
rationale = ap.sql_literal(str(child["rationale"]))
|
||||||
|
payload = ap.sql_literal(json.dumps(child["payload"], sort_keys=True))
|
||||||
|
return f"""\\set ON_ERROR_STOP on
|
||||||
|
begin;
|
||||||
|
create temporary table normalized_stage_result (created boolean not null) on commit drop;
|
||||||
|
with inserted as (
|
||||||
|
insert into kb_stage.kb_proposals
|
||||||
|
(id, proposal_type, status, proposed_by_handle, proposed_by_agent_id,
|
||||||
|
channel, source_ref, rationale, payload)
|
||||||
|
values
|
||||||
|
({proposal_id}::uuid, {proposal_type}, 'pending_review', nullif({proposed_by_handle}, ''),
|
||||||
|
{agent_sql}, {channel}, {source_ref}, {rationale}, {payload}::jsonb)
|
||||||
|
on conflict (id) do nothing
|
||||||
|
returning id
|
||||||
|
)
|
||||||
|
insert into normalized_stage_result(created)
|
||||||
|
select exists(select 1 from inserted);
|
||||||
|
|
||||||
|
do $normalized_stage$
|
||||||
|
declare
|
||||||
|
marker_count integer;
|
||||||
|
exact_count integer;
|
||||||
|
begin
|
||||||
|
select count(*) into marker_count
|
||||||
|
from kb_stage.kb_proposals
|
||||||
|
where source_ref = {source_ref};
|
||||||
|
select count(*) into exact_count
|
||||||
|
from kb_stage.kb_proposals
|
||||||
|
where id = {proposal_id}::uuid
|
||||||
|
and proposal_type = {proposal_type}
|
||||||
|
and status = 'pending_review'
|
||||||
|
and proposed_by_handle is not distinct from nullif({proposed_by_handle}, '')
|
||||||
|
and proposed_by_agent_id is not distinct from {agent_sql}
|
||||||
|
and channel = {channel}
|
||||||
|
and source_ref = {source_ref}
|
||||||
|
and rationale = {rationale}
|
||||||
|
and payload = {payload}::jsonb
|
||||||
|
and reviewed_by_handle is null
|
||||||
|
and reviewed_by_agent_id is null
|
||||||
|
and reviewed_at is null
|
||||||
|
and review_note is null
|
||||||
|
and applied_by_handle is null
|
||||||
|
and applied_by_agent_id is null
|
||||||
|
and applied_at is null;
|
||||||
|
if marker_count <> 1 or exact_count <> 1 then
|
||||||
|
raise exception 'normalized stage validation failed: marker_count=%, exact_count=%',
|
||||||
|
marker_count, exact_count;
|
||||||
|
end if;
|
||||||
|
end
|
||||||
|
$normalized_stage$;
|
||||||
|
|
||||||
|
select jsonb_build_object(
|
||||||
|
'created', (select created from normalized_stage_result),
|
||||||
|
'proposal_id', id::text,
|
||||||
|
'proposal_type', proposal_type,
|
||||||
|
'status', status,
|
||||||
|
'source_ref', source_ref,
|
||||||
|
'payload_sha256', {ap.sql_literal(str(child["payload_sha256"]))}
|
||||||
|
)::text
|
||||||
|
from kb_stage.kb_proposals
|
||||||
|
where id = {proposal_id}::uuid;
|
||||||
|
commit;
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
def stage_normalized_child(container: str, database: str, parent: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
target = bound.container_identity(container)
|
||||||
|
production = bound.container_identity(bound.PRODUCTION_CONTAINER)
|
||||||
|
bound.validate_disposable_target_identity(target, production)
|
||||||
|
if target["id"] == production["id"]:
|
||||||
|
raise bound.CheckpointError("normalized staging target resolves to production")
|
||||||
|
child = prepare_normalized_child(parent)
|
||||||
|
result = bound._psql_json(str(target["id"]), database, build_stage_sql(child))
|
||||||
|
if result.get("proposal_id") != child["id"] or result.get("status") != "pending_review":
|
||||||
|
raise bound.CheckpointError("normalized staging did not return the exact pending child")
|
||||||
|
return {"child": child, "database_receipt": result, "bound_container_id": str(target["id"])}
|
||||||
|
|
||||||
|
|
||||||
|
def parse_args(argv: list[str] | None = None) -> argparse.Namespace:
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
|
parser.add_argument("--container", required=True)
|
||||||
|
parser.add_argument("--db", required=True)
|
||||||
|
parser.add_argument("--proposal-file", required=True, type=Path)
|
||||||
|
return parser.parse_args(argv)
|
||||||
|
|
||||||
|
|
||||||
|
def main(argv: list[str] | None = None) -> int:
|
||||||
|
args = parse_args(argv)
|
||||||
|
try:
|
||||||
|
if args.container == bound.PRODUCTION_CONTAINER:
|
||||||
|
raise bound.CheckpointError("--container must be a disposable non-production target")
|
||||||
|
parent = json.loads(args.proposal_file.read_text(encoding="utf-8"))
|
||||||
|
if not isinstance(parent, dict):
|
||||||
|
raise bound.CheckpointError("--proposal-file must contain one JSON object")
|
||||||
|
result = stage_normalized_child(args.container, args.db, parent)
|
||||||
|
except (OSError, json.JSONDecodeError, bound.CheckpointError, SystemExit, ValueError) as exc:
|
||||||
|
print(json.dumps({"status": "rejected", "error": str(exc)}, sort_keys=True))
|
||||||
|
return 2
|
||||||
|
print(
|
||||||
|
json.dumps(
|
||||||
|
{
|
||||||
|
"status": "pending_review",
|
||||||
|
"proposal_id": result["child"]["id"],
|
||||||
|
"payload_sha256": result["child"]["payload_sha256"],
|
||||||
|
"created": result["database_receipt"].get("created"),
|
||||||
|
"bound_container_id": result["bound_container_id"],
|
||||||
|
},
|
||||||
|
sort_keys=True,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
168
tests/test_bootstrap_clone_kb_gate.py
Normal file
168
tests/test_bootstrap_clone_kb_gate.py
Normal file
|
|
@ -0,0 +1,168 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
REPO_ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
sys.path.insert(0, str(REPO_ROOT / "scripts"))
|
||||||
|
|
||||||
|
import bootstrap_clone_kb_gate as bootstrap # noqa: E402
|
||||||
|
|
||||||
|
|
||||||
|
def _identity(name: str, identifier: str, *, disposable: bool) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"id": identifier,
|
||||||
|
"name": name,
|
||||||
|
"image": "postgres:16",
|
||||||
|
"labels": (
|
||||||
|
{bootstrap.bound.DISPOSABLE_LABEL_KEY: bootstrap.bound.DISPOSABLE_LABEL_VALUE}
|
||||||
|
if disposable
|
||||||
|
else {}
|
||||||
|
),
|
||||||
|
"network_mode": "none" if disposable else "bridge",
|
||||||
|
"ports": {},
|
||||||
|
"mount_sources": [f"/{name}-data"],
|
||||||
|
"started_at": f"{name}-start",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_bootstrap_installs_gate_verifies_logins_and_writes_only_private_secret_files(
|
||||||
|
monkeypatch: pytest.MonkeyPatch,
|
||||||
|
tmp_path: Path,
|
||||||
|
) -> None:
|
||||||
|
tmp_path.chmod(0o700)
|
||||||
|
target = _identity("clone", "clone-id", disposable=True)
|
||||||
|
production = _identity(bootstrap.bound.PRODUCTION_CONTAINER, "production-id", disposable=False)
|
||||||
|
monkeypatch.setattr(
|
||||||
|
bootstrap.bound,
|
||||||
|
"container_identity",
|
||||||
|
lambda name: dict(production if name == bootstrap.bound.PRODUCTION_CONTAINER else target),
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(
|
||||||
|
bootstrap.bound,
|
||||||
|
"validate_disposable_target_identity",
|
||||||
|
lambda observed, live: {"all": observed["id"] != live["id"], "network_none": True},
|
||||||
|
)
|
||||||
|
passwords = iter(("review-test-secret", "apply-test-secret"))
|
||||||
|
monkeypatch.setattr(bootstrap.secrets, "token_urlsafe", lambda _length: next(passwords))
|
||||||
|
sql_calls: list[dict[str, Any]] = []
|
||||||
|
monkeypatch.setattr(
|
||||||
|
bootstrap,
|
||||||
|
"_psql",
|
||||||
|
lambda container, database, sql, **kwargs: sql_calls.append(
|
||||||
|
{"container": container, "database": database, "sql": sql, **kwargs}
|
||||||
|
),
|
||||||
|
)
|
||||||
|
login_calls: list[tuple[str, str, str, str]] = []
|
||||||
|
monkeypatch.setattr(
|
||||||
|
bootstrap,
|
||||||
|
"_verify_login",
|
||||||
|
lambda container, database, role, password: login_calls.append((container, database, role, password)),
|
||||||
|
)
|
||||||
|
manifest = {
|
||||||
|
"approval_table": "kb_stage.kb_proposal_approvals",
|
||||||
|
"review_principals_table": "kb_stage.kb_review_principals",
|
||||||
|
"review_role_exists": True,
|
||||||
|
"apply_role_exists": True,
|
||||||
|
}
|
||||||
|
monkeypatch.setattr(bootstrap.bound, "_psql_json", lambda *_args, **_kwargs: dict(manifest))
|
||||||
|
|
||||||
|
result = bootstrap.bootstrap("clone", "teleo", tmp_path)
|
||||||
|
|
||||||
|
assert result["status"] == "ready"
|
||||||
|
assert result["bound_container_id"] == "clone-id"
|
||||||
|
assert result["gate_manifest"] == manifest
|
||||||
|
assert result["secret_values_or_hashes_recorded"] is False
|
||||||
|
assert login_calls == [
|
||||||
|
("clone-id", "teleo", "kb_review", "review-test-secret"),
|
||||||
|
("clone-id", "teleo", "kb_apply", "apply-test-secret"),
|
||||||
|
]
|
||||||
|
assert len(sql_calls) == 3
|
||||||
|
assert sql_calls[-1]["secret_values"] == ("review-test-secret", "apply-test-secret")
|
||||||
|
review_file = tmp_path / "kb-review.env"
|
||||||
|
apply_file = tmp_path / "kb-apply.env"
|
||||||
|
assert review_file.read_text(encoding="utf-8") == "KB_REVIEW_PASSWORD=review-test-secret\n"
|
||||||
|
assert apply_file.read_text(encoding="utf-8") == "KB_APPLY_DB_PASSWORD=apply-test-secret\n"
|
||||||
|
assert review_file.stat().st_mode & 0o777 == 0o600
|
||||||
|
assert apply_file.stat().st_mode & 0o777 == 0o600
|
||||||
|
serialized = json.dumps(result, sort_keys=True)
|
||||||
|
assert "review-test-secret" not in serialized
|
||||||
|
assert "apply-test-secret" not in serialized
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("mode", [0o755, 0o750])
|
||||||
|
def test_bootstrap_rejects_nonprivate_output_directory(
|
||||||
|
monkeypatch: pytest.MonkeyPatch,
|
||||||
|
tmp_path: Path,
|
||||||
|
mode: int,
|
||||||
|
) -> None:
|
||||||
|
tmp_path.chmod(mode)
|
||||||
|
target = _identity("clone", "clone-id", disposable=True)
|
||||||
|
production = _identity(bootstrap.bound.PRODUCTION_CONTAINER, "production-id", disposable=False)
|
||||||
|
monkeypatch.setattr(
|
||||||
|
bootstrap.bound,
|
||||||
|
"container_identity",
|
||||||
|
lambda name: dict(production if name == bootstrap.bound.PRODUCTION_CONTAINER else target),
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(bootstrap.bound, "validate_disposable_target_identity", lambda *_args: {"all": True})
|
||||||
|
|
||||||
|
with pytest.raises(bootstrap.bound.CheckpointError, match="private directory"):
|
||||||
|
bootstrap.bootstrap("clone", "teleo", tmp_path)
|
||||||
|
|
||||||
|
|
||||||
|
def test_bootstrap_refuses_existing_or_symlinked_secret_paths(
|
||||||
|
monkeypatch: pytest.MonkeyPatch,
|
||||||
|
tmp_path: Path,
|
||||||
|
) -> None:
|
||||||
|
tmp_path.chmod(0o700)
|
||||||
|
target = _identity("clone", "clone-id", disposable=True)
|
||||||
|
production = _identity(bootstrap.bound.PRODUCTION_CONTAINER, "production-id", disposable=False)
|
||||||
|
monkeypatch.setattr(
|
||||||
|
bootstrap.bound,
|
||||||
|
"container_identity",
|
||||||
|
lambda name: dict(production if name == bootstrap.bound.PRODUCTION_CONTAINER else target),
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(bootstrap.bound, "validate_disposable_target_identity", lambda *_args: {"all": True})
|
||||||
|
(tmp_path / "kb-review.env").symlink_to(tmp_path / "missing-target")
|
||||||
|
|
||||||
|
with pytest.raises(bootstrap.bound.CheckpointError, match="already exist"):
|
||||||
|
bootstrap.bootstrap("clone", "teleo", tmp_path)
|
||||||
|
|
||||||
|
|
||||||
|
def test_write_secret_is_exclusive_and_does_not_follow_symlinks(tmp_path: Path) -> None:
|
||||||
|
target = tmp_path / "target"
|
||||||
|
target.write_text("unchanged\n", encoding="utf-8")
|
||||||
|
link = tmp_path / "secret.env"
|
||||||
|
link.symlink_to(target)
|
||||||
|
|
||||||
|
with pytest.raises(FileExistsError):
|
||||||
|
bootstrap._write_secret(link, "SECRET", "value")
|
||||||
|
assert target.read_text(encoding="utf-8") == "unchanged\n"
|
||||||
|
|
||||||
|
|
||||||
|
def test_main_rejects_production_without_calling_bootstrap(
|
||||||
|
monkeypatch: pytest.MonkeyPatch,
|
||||||
|
tmp_path: Path,
|
||||||
|
capsys: pytest.CaptureFixture[str],
|
||||||
|
) -> None:
|
||||||
|
called = False
|
||||||
|
|
||||||
|
def unexpected(*_args: Any, **_kwargs: Any) -> dict[str, Any]:
|
||||||
|
nonlocal called
|
||||||
|
called = True
|
||||||
|
return {}
|
||||||
|
|
||||||
|
monkeypatch.setattr(bootstrap, "bootstrap", unexpected)
|
||||||
|
result = bootstrap.main(
|
||||||
|
["--container", bootstrap.bound.PRODUCTION_CONTAINER, "--db", "teleo", "--output-dir", str(tmp_path)]
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result == 2
|
||||||
|
assert called is False
|
||||||
|
output = json.loads(capsys.readouterr().out)
|
||||||
|
assert output["status"] == "rejected"
|
||||||
|
assert "non-production" in output["error"]
|
||||||
|
|
@ -194,6 +194,71 @@ def test_rich_claim_graph_normalizes_to_one_atomic_approve_claim_bundle():
|
||||||
assert accounting["total_planned_source_rows"] == 3
|
assert accounting["total_planned_source_rows"] == 3
|
||||||
|
|
||||||
|
|
||||||
|
def test_source_content_sha256_is_preserved_as_the_canonical_source_hash():
|
||||||
|
content_hash = "ab" * 32
|
||||||
|
preview = norm.normalize_proposal(
|
||||||
|
{
|
||||||
|
"id": "77777777-7777-4777-8777-777777777779",
|
||||||
|
"proposal_type": "attach_evidence",
|
||||||
|
"status": "pending_review",
|
||||||
|
"payload": {
|
||||||
|
"claim_candidates": [
|
||||||
|
{
|
||||||
|
"claim_key": "hash_bound_claim",
|
||||||
|
"text": "The claim is bound to source bytes.",
|
||||||
|
"evidence": [{"source_key": "hash_bound_source", "role": "grounds"}],
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"source_candidates": [
|
||||||
|
{
|
||||||
|
"source_key": "hash_bound_source",
|
||||||
|
"source_type": "document",
|
||||||
|
"storage_path": "/evidence/source.txt",
|
||||||
|
"key_excerpt": "The claim is bound to source bytes.",
|
||||||
|
"content_sha256": content_hash.upper(),
|
||||||
|
}
|
||||||
|
],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
assert preview["normalization_state"] == "strict_children_ready"
|
||||||
|
source_rows = preview["strict_child_proposals"][0]["payload"]["apply_payload"]["sources"]
|
||||||
|
source = next(row for row in source_rows if row["storage_path"] == "/evidence/source.txt")
|
||||||
|
assert source["hash"] == content_hash
|
||||||
|
assert content_hash.upper() in source["excerpt"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_invalid_source_content_hash_blocks_rich_normalization():
|
||||||
|
preview = norm.normalize_proposal(
|
||||||
|
{
|
||||||
|
"id": "77777777-7777-4777-8777-777777777780",
|
||||||
|
"proposal_type": "attach_evidence",
|
||||||
|
"status": "pending_review",
|
||||||
|
"payload": {
|
||||||
|
"claim_candidates": [
|
||||||
|
{
|
||||||
|
"claim_key": "bad_hash_claim",
|
||||||
|
"text": "This must not normalize.",
|
||||||
|
"evidence": [{"source_key": "bad_hash_source"}],
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"source_candidates": [
|
||||||
|
{
|
||||||
|
"source_key": "bad_hash_source",
|
||||||
|
"key_excerpt": "This must not normalize.",
|
||||||
|
"content_sha256": "not-a-sha256",
|
||||||
|
}
|
||||||
|
],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
assert preview["normalization_state"] == "blocked_lossy_rich_normalization"
|
||||||
|
assert preview["strict_child_count"] == 0
|
||||||
|
assert any("invalid content_sha256" in item["reason"] for item in preview["blocked_fragments"])
|
||||||
|
|
||||||
|
|
||||||
def test_rich_normalization_blocks_malformed_or_duplicate_candidates():
|
def test_rich_normalization_blocks_malformed_or_duplicate_candidates():
|
||||||
proposal = {
|
proposal = {
|
||||||
"id": "77777777-7777-4777-8777-777777777777",
|
"id": "77777777-7777-4777-8777-777777777777",
|
||||||
|
|
@ -247,17 +312,11 @@ def test_rich_normalization_retains_type_mapping_manifest_for_review():
|
||||||
)
|
)
|
||||||
child = preview["strict_child_proposals"][0]
|
child = preview["strict_child_proposals"][0]
|
||||||
manifest = child["payload"]["apply_payload"]["normalization_manifest"]
|
manifest = child["payload"]["apply_payload"]["normalization_manifest"]
|
||||||
assert {
|
assert {(item["claim_key"], item["from"], item["to"]) for item in manifest["claim_type_mappings"]} == {
|
||||||
(item["claim_key"], item["from"], item["to"])
|
|
||||||
for item in manifest["claim_type_mappings"]
|
|
||||||
} == {
|
|
||||||
("governance_claim", "governance", "meta"),
|
("governance_claim", "governance", "meta"),
|
||||||
("typed_claim", "normative/detail", "normative"),
|
("typed_claim", "normative/detail", "normative"),
|
||||||
}
|
}
|
||||||
assert {
|
assert {(item["source_key"], item["from"], item["to"]) for item in manifest["source_type_mappings"]} == {
|
||||||
(item["source_key"], item["from"], item["to"])
|
|
||||||
for item in manifest["source_type_mappings"]
|
|
||||||
} == {
|
|
||||||
("book_source", "book", "other"),
|
("book_source", "book", "other"),
|
||||||
("web_source", "url", "article"),
|
("web_source", "url", "article"),
|
||||||
("governance_claim__proposal_body", "proposal_body", "other"),
|
("governance_claim__proposal_body", "proposal_body", "other"),
|
||||||
|
|
|
||||||
1174
tests/test_run_leo_clone_bound_handler_checkpoint.py
Normal file
1174
tests/test_run_leo_clone_bound_handler_checkpoint.py
Normal file
File diff suppressed because it is too large
Load diff
738
tests/test_run_leo_clone_composition_checkpoint.py
Normal file
738
tests/test_run_leo_clone_composition_checkpoint.py
Normal file
|
|
@ -0,0 +1,738 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import asyncio
|
||||||
|
import copy
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
REPO_ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
sys.path.insert(0, str(REPO_ROOT / "scripts"))
|
||||||
|
|
||||||
|
import run_leo_clone_composition_checkpoint as composition # noqa: E402
|
||||||
|
|
||||||
|
RUN_MARKER = "composition-unit-20260711"
|
||||||
|
CONVERSATION_MARKER = "composition-memory-unit-20260711"
|
||||||
|
AGENT_ID = "11111111-1111-4111-8111-111111111111"
|
||||||
|
|
||||||
|
|
||||||
|
def _packet(fixture: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"claim_candidates": [
|
||||||
|
{
|
||||||
|
"claim_key": "guarded_apply_required",
|
||||||
|
"text": "Approval leaves canonical rows unchanged until guarded apply succeeds.",
|
||||||
|
"type": "structural",
|
||||||
|
"confidence": 0.98,
|
||||||
|
"evidence": [{"source_key": "operating_document", "role": "grounds"}],
|
||||||
|
"edges": [
|
||||||
|
{
|
||||||
|
"edge_type": "contradicts",
|
||||||
|
"target": "approval_is_canonical",
|
||||||
|
}
|
||||||
|
],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"claim_key": "approval_is_canonical",
|
||||||
|
"text": "Reviewer approval immediately makes a Leo proposal canonical.",
|
||||||
|
"type": "empirical",
|
||||||
|
"confidence": 0.4,
|
||||||
|
"evidence": [{"source_key": "contradictory_post", "role": "grounds"}],
|
||||||
|
},
|
||||||
|
],
|
||||||
|
"source_candidates": [
|
||||||
|
{
|
||||||
|
"source_key": "operating_document",
|
||||||
|
"source_type": "article",
|
||||||
|
"storage_path": fixture["document"]["storage_path"],
|
||||||
|
"url": None,
|
||||||
|
"title": fixture["document"]["title"],
|
||||||
|
"source_quality": "internal_operating_document",
|
||||||
|
"key_excerpt": fixture["document"]["text"],
|
||||||
|
"content_sha256": fixture["document"]["content_sha256"],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"source_key": "contradictory_post",
|
||||||
|
"source_type": "post",
|
||||||
|
"storage_path": None,
|
||||||
|
"url": fixture["post"]["url"],
|
||||||
|
"title": fixture["post"]["title"],
|
||||||
|
"source_quality": "unverified_social_post",
|
||||||
|
"key_excerpt": fixture["post"]["text"],
|
||||||
|
"content_sha256": fixture["post"]["content_sha256"],
|
||||||
|
},
|
||||||
|
],
|
||||||
|
"dedupe_conflict_assessment": {
|
||||||
|
"database_search_query": fixture["project"],
|
||||||
|
"potential_existing_claim_ids": [],
|
||||||
|
"conflicts": [
|
||||||
|
{
|
||||||
|
"from_claim_key": "guarded_apply_required",
|
||||||
|
"to_claim_key": "approval_is_canonical",
|
||||||
|
"relationship": "contradicts",
|
||||||
|
}
|
||||||
|
],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_fixture_and_prompts_keep_source_bytes_and_memory_boundaries_explicit() -> None:
|
||||||
|
fixture = composition.build_source_fixture(RUN_MARKER)
|
||||||
|
prompts = composition.build_prompts(fixture, CONVERSATION_MARKER)
|
||||||
|
|
||||||
|
assert fixture["project"] in fixture["document"]["text"]
|
||||||
|
assert fixture["project"] in fixture["post"]["text"]
|
||||||
|
assert fixture["document"]["content_sha256"] in prompts["extract"]
|
||||||
|
assert fixture["post"]["content_sha256"] in prompts["extract"]
|
||||||
|
assert CONVERSATION_MARKER in prompts["extract"]
|
||||||
|
assert CONVERSATION_MARKER not in prompts["reason"]
|
||||||
|
assert fixture["document"]["text"] not in prompts["reason"]
|
||||||
|
assert fixture["post"]["text"] not in prompts["reason"]
|
||||||
|
child = composition.normalized_stage.prepare_normalized_child(
|
||||||
|
composition.build_parent_packet(_packet(fixture), fixture, RUN_MARKER, AGENT_ID)
|
||||||
|
)
|
||||||
|
assert child["id"] not in prompts["reason"]
|
||||||
|
assert all(
|
||||||
|
row["id"] not in prompts["reason"]
|
||||||
|
for collection in ("claims", "sources")
|
||||||
|
for row in child["payload"]["apply_payload"][collection]
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_valid_packet_is_hash_bound_evidenced_and_conflict_aware() -> None:
|
||||||
|
fixture = composition.build_source_fixture(RUN_MARKER)
|
||||||
|
validation = composition.validate_composition_packet(_packet(fixture), fixture)
|
||||||
|
|
||||||
|
assert validation["claim_count"] == 2
|
||||||
|
assert validation["source_count"] == 2
|
||||||
|
assert validation["evidence_source_keys"] == ["contradictory_post", "operating_document"]
|
||||||
|
assert validation["conflict_pairs"] == [["guarded_apply_required", "approval_is_canonical"]]
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("mutation", ["hash", "excerpt", "evidence", "conflict"])
|
||||||
|
def test_packet_validation_fails_closed_on_provenance_or_reasoning_loss(mutation: str) -> None:
|
||||||
|
fixture = composition.build_source_fixture(RUN_MARKER)
|
||||||
|
packet = _packet(fixture)
|
||||||
|
if mutation == "hash":
|
||||||
|
packet["source_candidates"][0]["content_sha256"] = "0" * 64
|
||||||
|
elif mutation == "excerpt":
|
||||||
|
packet["source_candidates"][0]["key_excerpt"] = "hallucinated excerpt"
|
||||||
|
elif mutation == "evidence":
|
||||||
|
packet["claim_candidates"][0]["evidence"] = []
|
||||||
|
else:
|
||||||
|
packet["claim_candidates"][0]["edges"] = []
|
||||||
|
|
||||||
|
with pytest.raises(composition.bound.CheckpointError):
|
||||||
|
composition.validate_composition_packet(packet, fixture)
|
||||||
|
|
||||||
|
|
||||||
|
def test_packet_normalizes_to_deterministic_hash_bound_claim_graph() -> None:
|
||||||
|
fixture = composition.build_source_fixture(RUN_MARKER)
|
||||||
|
packet = _packet(fixture)
|
||||||
|
parent = composition.build_parent_packet(packet, fixture, RUN_MARKER, AGENT_ID)
|
||||||
|
child = composition.normalized_stage.prepare_normalized_child(parent)
|
||||||
|
payload = child["payload"]["apply_payload"]
|
||||||
|
|
||||||
|
assert parent["id"] == composition._stable_uuid(RUN_MARKER, "rich-source-packet")
|
||||||
|
assert len(payload["claims"]) == 2
|
||||||
|
assert len(payload["sources"]) == 4
|
||||||
|
assert len(payload["evidence"]) == 4
|
||||||
|
assert len(payload["edges"]) == 1
|
||||||
|
assert payload["edges"][0]["edge_type"] == "contradicts"
|
||||||
|
assert {fixture["document"]["content_sha256"], fixture["post"]["content_sha256"]} <= {
|
||||||
|
row["hash"] for row in payload["sources"]
|
||||||
|
}
|
||||||
|
assert composition.expected_table_deltas(child) == {
|
||||||
|
"public.claims": 2,
|
||||||
|
"public.sources": 4,
|
||||||
|
"public.claim_evidence": 4,
|
||||||
|
"public.claim_edges": 1,
|
||||||
|
"public.reasoning_tools": 0,
|
||||||
|
"kb_stage.kb_proposals": 1,
|
||||||
|
composition.lifecycle.APPROVAL_TABLE: 1,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_final_state_receipt_must_match_every_generated_row_id() -> None:
|
||||||
|
fixture = composition.build_source_fixture(RUN_MARKER)
|
||||||
|
child = composition.normalized_stage.prepare_normalized_child(
|
||||||
|
composition.build_parent_packet(_packet(fixture), fixture, RUN_MARKER, AGENT_ID)
|
||||||
|
)
|
||||||
|
payload = child["payload"]["apply_payload"]
|
||||||
|
edge = payload["edges"][0]
|
||||||
|
state = {
|
||||||
|
"phase": "final",
|
||||||
|
"conversation_marker": CONVERSATION_MARKER,
|
||||||
|
"proposal_id": child["id"],
|
||||||
|
"status": "applied",
|
||||||
|
"claim_ids": [row["id"] for row in payload["claims"]],
|
||||||
|
"source_ids": [row["id"] for row in payload["sources"]],
|
||||||
|
"conflict_edge": {
|
||||||
|
"from_claim": edge["from_claim"],
|
||||||
|
"to_claim": edge["to_claim"],
|
||||||
|
"edge_type": edge["edge_type"],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
assert composition._state_matches(state, conversation_marker=CONVERSATION_MARKER, child=child) is True
|
||||||
|
marker_alias = copy.deepcopy(state)
|
||||||
|
marker_alias["marker"] = marker_alias.pop("conversation_marker")
|
||||||
|
assert composition._state_matches(marker_alias, conversation_marker=CONVERSATION_MARKER, child=child) is True
|
||||||
|
wrong = copy.deepcopy(state)
|
||||||
|
wrong["source_ids"] = wrong["source_ids"][:-1]
|
||||||
|
assert composition._state_matches(wrong, conversation_marker=CONVERSATION_MARKER, child=child) is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_unrelated_guard_excludes_every_planned_row_and_is_read_only(
|
||||||
|
monkeypatch: pytest.MonkeyPatch,
|
||||||
|
) -> None:
|
||||||
|
fixture = composition.build_source_fixture(RUN_MARKER)
|
||||||
|
child = composition.normalized_stage.prepare_normalized_child(
|
||||||
|
composition.build_parent_packet(_packet(fixture), fixture, RUN_MARKER, AGENT_ID)
|
||||||
|
)
|
||||||
|
observed: dict[str, str] = {}
|
||||||
|
|
||||||
|
def fake_psql(_container: str, _database: str, sql: str) -> dict[str, Any]:
|
||||||
|
observed["sql"] = sql
|
||||||
|
return {table: {"count": 1, "rowset_md5": "same"} for table in composition.TABLES}
|
||||||
|
|
||||||
|
monkeypatch.setattr(composition.bound, "_psql_json", fake_psql)
|
||||||
|
snapshot = composition.unrelated_guard_snapshot("clone-id", "teleo", child)
|
||||||
|
lowered = observed["sql"].lower()
|
||||||
|
|
||||||
|
assert set(snapshot) == set(composition.TABLES)
|
||||||
|
assert "begin transaction read only" in lowered
|
||||||
|
assert child["id"] in observed["sql"]
|
||||||
|
for collection in ("claims", "sources"):
|
||||||
|
for row in child["payload"]["apply_payload"][collection]:
|
||||||
|
assert row["id"] in observed["sql"]
|
||||||
|
assert "insert into" not in lowered
|
||||||
|
assert "update " not in lowered
|
||||||
|
assert "delete from" not in lowered
|
||||||
|
|
||||||
|
|
||||||
|
def test_prefixed_receipts_are_exact_json_objects() -> None:
|
||||||
|
payload = {"claim_candidates": [], "source_candidates": []}
|
||||||
|
reply = "analysis\nCOMPOSITION_PACKET: " + json.dumps(payload, sort_keys=True)
|
||||||
|
|
||||||
|
assert composition.parse_prefixed_json(reply, composition.PACKET_PREFIX) == payload
|
||||||
|
with pytest.raises(composition.bound.CheckpointError, match="invalid"):
|
||||||
|
composition.parse_prefixed_json("COMPOSITION_PACKET: nope", composition.PACKET_PREFIX)
|
||||||
|
with pytest.raises(composition.bound.CheckpointError, match="did not contain"):
|
||||||
|
composition.parse_prefixed_json("no receipt", composition.PACKET_PREFIX)
|
||||||
|
|
||||||
|
|
||||||
|
def test_semantic_search_and_exact_read_helpers_require_database_commands_without_generated_ids() -> None:
|
||||||
|
claim_id = "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa"
|
||||||
|
proposal_id = "bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb"
|
||||||
|
project = "AURORA-TEST"
|
||||||
|
turn = {
|
||||||
|
"tool_invocations": [
|
||||||
|
{"argv": ["search", "approval apply canonical guarded operator review"]},
|
||||||
|
{"argv": ["search-proposals", project, "--status", "all", "--limit", "20"]},
|
||||||
|
{"argv": ["show-proposal", proposal_id]},
|
||||||
|
{"argv": ["show", claim_id]},
|
||||||
|
{"argv": ["evidence", claim_id, "--format", "json"]},
|
||||||
|
{"argv": ["edges", claim_id, "--format", "json"]},
|
||||||
|
]
|
||||||
|
}
|
||||||
|
|
||||||
|
assert composition._searched_semantically_without_ids(turn, [claim_id, proposal_id]) is True
|
||||||
|
assert composition._searched_composed_proposal(turn, project) is True
|
||||||
|
assert composition._read_exact_proposal(turn, proposal_id) is True
|
||||||
|
assert composition._read_claim(turn, claim_id) is True
|
||||||
|
assert composition._read_claim_evidence(turn, claim_id) is True
|
||||||
|
assert composition._read_claim_edges(turn, claim_id) is True
|
||||||
|
evidence_only = {"tool_invocations": [{"argv": ["evidence", claim_id, "--format", "json"]}]}
|
||||||
|
assert composition._read_claim_material(evidence_only, claim_id) is True
|
||||||
|
turn["tool_invocations"][0]["argv"][1] = f"show {claim_id}"
|
||||||
|
assert composition._searched_semantically_without_ids(turn, [claim_id, proposal_id]) is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_cumulative_transcript_merge_accepts_only_receipted_truncated_unknown_results() -> None:
|
||||||
|
first = [
|
||||||
|
{"phase": "call", "tool_call_id": "call-1", "tool_name": "terminal"},
|
||||||
|
{
|
||||||
|
"phase": "result",
|
||||||
|
"tool_call_id": "call-1",
|
||||||
|
"tool_name": "terminal",
|
||||||
|
"content": '{"exit_code":0}',
|
||||||
|
"content_truncated": False,
|
||||||
|
},
|
||||||
|
]
|
||||||
|
second = [
|
||||||
|
*first,
|
||||||
|
{"phase": "call", "tool_call_id": "call-2", "tool_name": "terminal"},
|
||||||
|
{
|
||||||
|
"phase": "result",
|
||||||
|
"tool_call_id": "call-2",
|
||||||
|
"tool_name": "terminal",
|
||||||
|
"content": '{"output":"large',
|
||||||
|
"content_truncated": True,
|
||||||
|
},
|
||||||
|
]
|
||||||
|
gateways = [
|
||||||
|
{"transcript_tool_trace": {"events": first}},
|
||||||
|
{"transcript_tool_trace": {"events": second}},
|
||||||
|
]
|
||||||
|
|
||||||
|
merged = composition.merge_transcript_events(gateways)
|
||||||
|
consistency = composition.terminal_trace_consistency(
|
||||||
|
merged,
|
||||||
|
{
|
||||||
|
"invocation_count": 2,
|
||||||
|
"all_bound_to_supplied_target": True,
|
||||||
|
"all_completed_successfully": True,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert len(merged) == 4
|
||||||
|
assert consistency["passes"] is True
|
||||||
|
assert consistency["unknown_result_call_ids"] == ["call-2"]
|
||||||
|
missing_receipt = [event for event in merged if event.get("tool_call_id") != "call-2" or event["phase"] != "result"]
|
||||||
|
assert composition.terminal_trace_consistency(
|
||||||
|
missing_receipt,
|
||||||
|
{
|
||||||
|
"invocation_count": 2,
|
||||||
|
"all_bound_to_supplied_target": True,
|
||||||
|
"all_completed_successfully": True,
|
||||||
|
},
|
||||||
|
)["passes"] is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_source_contains_no_production_write_restart_or_delivery_adapter() -> None:
|
||||||
|
source = (REPO_ROOT / "scripts" / "run_leo_clone_composition_checkpoint.py").read_text(encoding="utf-8").lower()
|
||||||
|
|
||||||
|
assert "insert into public." not in source
|
||||||
|
assert "update public." not in source
|
||||||
|
assert "delete from public." not in source
|
||||||
|
assert "systemctl restart" not in source
|
||||||
|
assert "send_message(" not in source
|
||||||
|
|
||||||
|
|
||||||
|
def test_source_manifest_covers_the_full_composition_and_gate_dependency_closure() -> None:
|
||||||
|
paths = {item["repo_relative_path"] for item in composition.source_manifest()}
|
||||||
|
|
||||||
|
assert paths == {
|
||||||
|
"scripts/apply_proposal.py",
|
||||||
|
"scripts/apply_worker.py",
|
||||||
|
"scripts/approve_proposal.py",
|
||||||
|
"scripts/bootstrap_clone_kb_gate.py",
|
||||||
|
"scripts/kb_apply_prereqs.sql",
|
||||||
|
"scripts/kb_proposal_normalize.py",
|
||||||
|
"scripts/kb_proposal_review_packet.py",
|
||||||
|
"scripts/kb_rich_proposal_creation_plan.py",
|
||||||
|
"scripts/run_approve_claim_clone_canary.py",
|
||||||
|
"scripts/run_leo_clone_bound_handler_checkpoint.py",
|
||||||
|
"scripts/run_leo_clone_composition_checkpoint.py",
|
||||||
|
"scripts/run_leo_clone_lifecycle_checkpoint.py",
|
||||||
|
"scripts/stage_normalized_proposal.py",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _args(tmp_path: Path) -> argparse.Namespace:
|
||||||
|
return argparse.Namespace(
|
||||||
|
container="composition-clone",
|
||||||
|
db="teleo",
|
||||||
|
output=tmp_path / "composition-report.json",
|
||||||
|
run_marker=RUN_MARKER,
|
||||||
|
conversation_marker=CONVERSATION_MARKER,
|
||||||
|
chat_id=composition.bound.DEFAULT_CHAT_ID,
|
||||||
|
user_id=composition.bound.DEFAULT_USER_ID,
|
||||||
|
user_name="composition unit",
|
||||||
|
turn_timeout=30,
|
||||||
|
review_secrets_file=str(tmp_path / "kb-review.env"),
|
||||||
|
apply_secrets_file=str(tmp_path / "kb-apply.env"),
|
||||||
|
copy_model_auth=True,
|
||||||
|
operator_review=True,
|
||||||
|
guarded_apply=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _proposal(child: dict[str, Any], state: str) -> dict[str, Any]:
|
||||||
|
reviewed = state in {"approved", "applied"}
|
||||||
|
applied = state == "applied"
|
||||||
|
return {
|
||||||
|
"id": child["id"],
|
||||||
|
"proposal_type": child["proposal_type"],
|
||||||
|
"status": state,
|
||||||
|
"channel": child["channel"],
|
||||||
|
"source_ref": child["source_ref"],
|
||||||
|
"rationale": child["rationale"],
|
||||||
|
"payload": child["payload"],
|
||||||
|
"reviewed_by_handle": composition.lifecycle.REVIEWER_HANDLE if reviewed else None,
|
||||||
|
"reviewed_by_agent_id": AGENT_ID if reviewed else None,
|
||||||
|
"reviewed_at": "2026-07-11T10:00:00+00:00" if reviewed else None,
|
||||||
|
"review_note": composition.lifecycle.REVIEW_NOTE if reviewed else None,
|
||||||
|
"applied_by_handle": composition.lifecycle.APPLIED_BY_HANDLE if applied else None,
|
||||||
|
"applied_by_agent_id": AGENT_ID if applied else None,
|
||||||
|
"applied_at": "2026-07-11T10:01:00+00:00" if applied else None,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _approval_snapshot(proposal: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"proposal_id": proposal["id"],
|
||||||
|
"proposal_type": proposal["proposal_type"],
|
||||||
|
"payload": proposal["payload"],
|
||||||
|
"reviewed_by_handle": proposal["reviewed_by_handle"],
|
||||||
|
"reviewed_by_agent_id": proposal["reviewed_by_agent_id"],
|
||||||
|
"reviewed_at": proposal["reviewed_at"],
|
||||||
|
"review_note": proposal["review_note"],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_mocked_full_composition_checkpoint_passes_with_exact_rows_and_cleanup(
|
||||||
|
monkeypatch: pytest.MonkeyPatch,
|
||||||
|
tmp_path: Path,
|
||||||
|
) -> None:
|
||||||
|
args = _args(tmp_path)
|
||||||
|
Path(args.review_secrets_file).write_text("KB_REVIEW_PASSWORD=test\n", encoding="utf-8")
|
||||||
|
Path(args.apply_secrets_file).write_text("KB_APPLY_PASSWORD=test\n", encoding="utf-8")
|
||||||
|
fixture = composition.build_source_fixture(args.run_marker)
|
||||||
|
packet = _packet(fixture)
|
||||||
|
parent = composition.build_parent_packet(packet, fixture, args.run_marker, AGENT_ID)
|
||||||
|
child = composition.normalized_stage.prepare_normalized_child(parent)
|
||||||
|
expected_rows = composition.guarded._expected_bundle_rows(child["payload"]["apply_payload"])
|
||||||
|
empty_rows = {key: [] for key in expected_rows}
|
||||||
|
base_counts = {table: 10 for table in composition.TABLES}
|
||||||
|
final_counts = {
|
||||||
|
table: base_counts[table] + composition.expected_table_deltas(child)[table] for table in composition.TABLES
|
||||||
|
}
|
||||||
|
base_snapshot = {
|
||||||
|
"counts": base_counts,
|
||||||
|
"rowset_md5": {table: f"base-{table}" for table in composition.TABLES},
|
||||||
|
}
|
||||||
|
state: dict[str, Any] = {"phase": "initial", "invocations": [], "transcript_events": []}
|
||||||
|
target_identity = {
|
||||||
|
"id": "composition-clone-id",
|
||||||
|
"name": args.container,
|
||||||
|
"image": "postgres",
|
||||||
|
"labels": {
|
||||||
|
composition.bound.DISPOSABLE_LABEL_KEY: composition.bound.DISPOSABLE_LABEL_VALUE,
|
||||||
|
},
|
||||||
|
"network_mode": "none",
|
||||||
|
"ports": {},
|
||||||
|
"mount_sources": ["/composition-clone-data"],
|
||||||
|
"started_at": "clone-start",
|
||||||
|
}
|
||||||
|
production_identity = {
|
||||||
|
"id": "production-id",
|
||||||
|
"name": composition.bound.PRODUCTION_CONTAINER,
|
||||||
|
"image": "postgres",
|
||||||
|
"labels": {},
|
||||||
|
"network_mode": "bridge",
|
||||||
|
"ports": {},
|
||||||
|
"mount_sources": ["/production-data"],
|
||||||
|
"started_at": "production-start",
|
||||||
|
}
|
||||||
|
monkeypatch.setattr(
|
||||||
|
composition.lifecycle,
|
||||||
|
"assert_disposable_target",
|
||||||
|
lambda _container, _database: (dict(target_identity), dict(production_identity)),
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(
|
||||||
|
composition.bound,
|
||||||
|
"container_identity",
|
||||||
|
lambda name: dict(production_identity if name == composition.bound.PRODUCTION_CONTAINER else target_identity),
|
||||||
|
)
|
||||||
|
|
||||||
|
def guard_snapshot(container: str, _database: str, **_kwargs: Any) -> dict[str, Any]:
|
||||||
|
snapshot = json.loads(json.dumps(base_snapshot))
|
||||||
|
production = container == composition.bound.PRODUCTION_CONTAINER
|
||||||
|
snapshot["database_identity"] = {
|
||||||
|
"current_database": args.db,
|
||||||
|
"system_identifier": "production-system" if production else "clone-system",
|
||||||
|
"transaction_read_only": "off",
|
||||||
|
}
|
||||||
|
if not production and state["phase"] == "applied":
|
||||||
|
snapshot["counts"] = dict(final_counts)
|
||||||
|
for table, delta in composition.expected_table_deltas(child).items():
|
||||||
|
if delta:
|
||||||
|
snapshot["rowset_md5"][table] = "changed"
|
||||||
|
return snapshot
|
||||||
|
|
||||||
|
monkeypatch.setattr(composition.bound, "database_guard_snapshot", guard_snapshot)
|
||||||
|
clone_gate = {
|
||||||
|
"approval_table": composition.lifecycle.APPROVAL_TABLE,
|
||||||
|
"review_principals_table": "kb_stage.kb_review_principals",
|
||||||
|
"review_role_exists": True,
|
||||||
|
"apply_role_exists": True,
|
||||||
|
"approve_function": "kb_stage.approve_strict_proposal",
|
||||||
|
"assert_function": "kb_stage.assert_approved_proposal",
|
||||||
|
"finish_function": "kb_stage.finish_approved_proposal",
|
||||||
|
}
|
||||||
|
production_gate = {
|
||||||
|
"approval_table": None,
|
||||||
|
"review_principals_table": None,
|
||||||
|
"review_role_exists": False,
|
||||||
|
"apply_role_exists": True,
|
||||||
|
"approve_function": None,
|
||||||
|
"assert_function": None,
|
||||||
|
"finish_function": None,
|
||||||
|
}
|
||||||
|
monkeypatch.setattr(
|
||||||
|
composition,
|
||||||
|
"gate_schema_manifest",
|
||||||
|
lambda container, _database: (
|
||||||
|
dict(production_gate) if container == composition.bound.PRODUCTION_CONTAINER else dict(clone_gate)
|
||||||
|
),
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(
|
||||||
|
composition.bound,
|
||||||
|
"service_state",
|
||||||
|
lambda: {"returncode": 0, "ActiveState": "active", "MainPID": "123", "NRestarts": "0"},
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(
|
||||||
|
composition.bound,
|
||||||
|
"bridge_hashes",
|
||||||
|
lambda: {"wrapper": "live-wrapper", "bridge_skill": "live-skill"},
|
||||||
|
)
|
||||||
|
runtime_root = tmp_path / "composition-runtime"
|
||||||
|
runtime_profile = runtime_root / "profile"
|
||||||
|
|
||||||
|
def copy_profile(**_kwargs: Any) -> tuple[Path, dict[str, Any]]:
|
||||||
|
runtime_profile.mkdir(parents=True)
|
||||||
|
composition.bound.register_temp_root(runtime_root)
|
||||||
|
return runtime_profile, {"passes": True, "secret_contents_read_or_recorded": False}
|
||||||
|
|
||||||
|
monkeypatch.setattr(composition.bound, "copy_profile", copy_profile)
|
||||||
|
monkeypatch.setattr(
|
||||||
|
composition.bound,
|
||||||
|
"copy_ephemeral_model_auth",
|
||||||
|
lambda _profile: {"mode": "ephemeral_file_copy", "source_contents_recorded": False},
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(
|
||||||
|
composition.bound,
|
||||||
|
"load_ephemeral_provider_environment",
|
||||||
|
lambda _profile: ({}, {"bound_credential_count": 0, "credential_contents_recorded": False}),
|
||||||
|
)
|
||||||
|
tool_log = runtime_profile / "tool-log.jsonl"
|
||||||
|
monkeypatch.setattr(
|
||||||
|
composition,
|
||||||
|
"patch_composition_bridge",
|
||||||
|
lambda _profile, **_kwargs: {
|
||||||
|
"tool_log_path": str(tool_log),
|
||||||
|
"run_nonce": "composition-run-nonce",
|
||||||
|
"wrapper_path": str(runtime_profile / "bin" / "teleo-kb"),
|
||||||
|
"bridge_skill_path": str(runtime_profile / "skills" / "teleo-kb-bridge" / "SKILL.md"),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(composition, "resolve_leo_agent_id", lambda _container, _database: AGENT_ID)
|
||||||
|
monkeypatch.setattr(
|
||||||
|
composition.normalized_stage,
|
||||||
|
"stage_normalized_child",
|
||||||
|
lambda _container, _database, _parent: (
|
||||||
|
state.update(phase="pending_review")
|
||||||
|
or {
|
||||||
|
"child": child,
|
||||||
|
"bound_container_id": target_identity["id"],
|
||||||
|
"database_receipt": {
|
||||||
|
"created": True,
|
||||||
|
"proposal_id": child["id"],
|
||||||
|
"proposal_type": child["proposal_type"],
|
||||||
|
"status": "pending_review",
|
||||||
|
"source_ref": child["source_ref"],
|
||||||
|
"payload_sha256": child["payload_sha256"],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
),
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(
|
||||||
|
composition.guarded,
|
||||||
|
"_exact_bundle_readback",
|
||||||
|
lambda _container, _database, _payload: expected_rows if state["phase"] == "applied" else empty_rows,
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(
|
||||||
|
composition.guarded,
|
||||||
|
"_proposal_readback",
|
||||||
|
lambda _container, _database, _proposal_id: (
|
||||||
|
None if state["phase"] == "initial" else _proposal(child, state["phase"])
|
||||||
|
),
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(
|
||||||
|
composition.guarded,
|
||||||
|
"_approval_snapshot_readback",
|
||||||
|
lambda _container, _database, _proposal_id: _approval_snapshot(_proposal(child, "approved")),
|
||||||
|
)
|
||||||
|
unrelated = {table: {"count": 10, "rowset_md5": f"unrelated-{table}"} for table in composition.TABLES}
|
||||||
|
monkeypatch.setattr(
|
||||||
|
composition,
|
||||||
|
"unrelated_guard_snapshot",
|
||||||
|
lambda _container, _database, _child: json.loads(json.dumps(unrelated)),
|
||||||
|
)
|
||||||
|
|
||||||
|
def review(
|
||||||
|
_args: argparse.Namespace, _proposal_id: str
|
||||||
|
) -> tuple[subprocess.CompletedProcess[str], subprocess.CompletedProcess[str]]:
|
||||||
|
state["phase"] = "approved"
|
||||||
|
return (
|
||||||
|
subprocess.CompletedProcess([], 0, "select kb_stage.approve_strict_proposal(...);", ""),
|
||||||
|
subprocess.CompletedProcess([], 0, "approved", ""),
|
||||||
|
)
|
||||||
|
|
||||||
|
def apply(
|
||||||
|
_args: argparse.Namespace, _proposal_id: str
|
||||||
|
) -> tuple[subprocess.CompletedProcess[str], subprocess.CompletedProcess[str]]:
|
||||||
|
state["phase"] = "applied"
|
||||||
|
return (
|
||||||
|
subprocess.CompletedProcess(
|
||||||
|
[],
|
||||||
|
0,
|
||||||
|
"select kb_stage.assert_approved_proposal(...); select kb_stage.finish_approved_proposal(...);",
|
||||||
|
"",
|
||||||
|
),
|
||||||
|
subprocess.CompletedProcess([], 0, "applied", ""),
|
||||||
|
)
|
||||||
|
|
||||||
|
monkeypatch.setattr(composition.lifecycle, "run_operator_review", review)
|
||||||
|
monkeypatch.setattr(composition.lifecycle, "run_guarded_apply", apply)
|
||||||
|
|
||||||
|
async def run_turn(
|
||||||
|
_args: argparse.Namespace,
|
||||||
|
_profile: Path,
|
||||||
|
*,
|
||||||
|
phase: str,
|
||||||
|
prompt: str,
|
||||||
|
provider_environment: dict[str, str],
|
||||||
|
tool_log: Path,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
assert provider_environment == {}
|
||||||
|
assert tool_log == runtime_profile / "tool-log.jsonl"
|
||||||
|
if phase == "T1_extract_sources":
|
||||||
|
reply = "Extracted two conflicting claims.\nCOMPOSITION_PACKET: " + json.dumps(
|
||||||
|
packet, sort_keys=True, separators=(",", ":")
|
||||||
|
)
|
||||||
|
argvs = [["context", packet["dedupe_conflict_assessment"]["database_search_query"]]]
|
||||||
|
pid = 101
|
||||||
|
else:
|
||||||
|
payload = child["payload"]["apply_payload"]
|
||||||
|
edge = payload["edges"][0]
|
||||||
|
receipt = {
|
||||||
|
"phase": "final",
|
||||||
|
"conversation_marker": args.conversation_marker,
|
||||||
|
"proposal_id": child["id"],
|
||||||
|
"status": "applied",
|
||||||
|
"claim_ids": [row["id"] for row in payload["claims"]],
|
||||||
|
"source_ids": [row["id"] for row in payload["sources"]],
|
||||||
|
"conflict_edge": {
|
||||||
|
"from_claim": edge["from_claim"],
|
||||||
|
"to_claim": edge["to_claim"],
|
||||||
|
"edge_type": edge["edge_type"],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
reply = (
|
||||||
|
"Approval was not canonical; guarded apply made it applied and canonical. The sources conflict. "
|
||||||
|
f"I recall {args.conversation_marker}.\nCOMPOSITION_STATE: "
|
||||||
|
+ json.dumps(receipt, sort_keys=True, separators=(",", ":"))
|
||||||
|
)
|
||||||
|
argvs = [
|
||||||
|
["search", "approval apply canonical guarded operator review"],
|
||||||
|
["search-proposals", fixture["project"], "--status", "all", "--limit", "20"],
|
||||||
|
["show-proposal", child["id"]],
|
||||||
|
*[["show", row["id"]] for row in payload["claims"]],
|
||||||
|
*[["evidence", row["id"], "--format", "json"] for row in payload["claims"]],
|
||||||
|
["edges", payload["claims"][0]["id"], "--format", "json"],
|
||||||
|
]
|
||||||
|
pid = 202
|
||||||
|
invocations = [
|
||||||
|
{
|
||||||
|
"invocation_id": f"{phase}-{index}",
|
||||||
|
"container": target_identity["id"],
|
||||||
|
"database": args.db,
|
||||||
|
"argv": argv,
|
||||||
|
"returncode": 0,
|
||||||
|
}
|
||||||
|
for index, argv in enumerate(argvs)
|
||||||
|
]
|
||||||
|
state["invocations"].extend(invocations)
|
||||||
|
current_events = [
|
||||||
|
event
|
||||||
|
for invocation in invocations
|
||||||
|
for event in (
|
||||||
|
{
|
||||||
|
"phase": "call",
|
||||||
|
"tool_name": "terminal",
|
||||||
|
"tool_call_id": invocation["invocation_id"],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"phase": "result",
|
||||||
|
"tool_name": "terminal",
|
||||||
|
"tool_call_id": invocation["invocation_id"],
|
||||||
|
"content": '{"exit_code":0}',
|
||||||
|
"content_truncated": False,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
]
|
||||||
|
state["transcript_events"].extend(current_events)
|
||||||
|
events = copy.deepcopy(state["transcript_events"])
|
||||||
|
return {
|
||||||
|
"phase": phase,
|
||||||
|
"prompt": prompt,
|
||||||
|
"prompt_sha256": "prompt-hash",
|
||||||
|
"tool_invocations": invocations,
|
||||||
|
"gateway": {
|
||||||
|
"authorized": True,
|
||||||
|
"handler_invoked": True,
|
||||||
|
"posted_to_telegram": False,
|
||||||
|
"model_free_fallback_used": False,
|
||||||
|
"reply": reply,
|
||||||
|
"session_key": "telegram:composition",
|
||||||
|
"transcript_tool_trace": {
|
||||||
|
"session_id": "persisted-composition-session",
|
||||||
|
"events": events,
|
||||||
|
},
|
||||||
|
"tool_surface": {
|
||||||
|
"allowed_tools": ["skills_list", "skill_view", "terminal"],
|
||||||
|
"actual_registry_tools": ["skills_list", "skill_view", "terminal"],
|
||||||
|
"gateway_adapters_verified_mapping": True,
|
||||||
|
"gateway_adapter_count": 0,
|
||||||
|
"send_message_tool_enabled": False,
|
||||||
|
"terminal_restricted_to_clone_wrapper": True,
|
||||||
|
"terminal_subprocess_inherits_provider_credentials": False,
|
||||||
|
},
|
||||||
|
"child_process": {
|
||||||
|
"pid": pid,
|
||||||
|
"readiness_verified": True,
|
||||||
|
"timed_out": False,
|
||||||
|
"exitcode": 0,
|
||||||
|
"result_transport": "private_temp_file",
|
||||||
|
"termination": {"attempted": False},
|
||||||
|
"alive_after_readback": False,
|
||||||
|
"process_group_alive_after_readback": False,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
monkeypatch.setattr(composition, "run_turn", run_turn)
|
||||||
|
monkeypatch.setattr(
|
||||||
|
composition.bound,
|
||||||
|
"read_tool_proof",
|
||||||
|
lambda _path, _container, _database, **_kwargs: {
|
||||||
|
"invocations": list(state["invocations"]),
|
||||||
|
"invocation_count": len(state["invocations"]),
|
||||||
|
"all_bound_to_supplied_target": bool(state["invocations"]),
|
||||||
|
"all_completed_successfully": bool(state["invocations"]),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
report = asyncio.run(composition.run_checkpoint(args))
|
||||||
|
|
||||||
|
assert report["status"] == "pass", {
|
||||||
|
"errors": report["errors"],
|
||||||
|
"failed_checks": [name for name, passed in report["checks"].items() if not passed],
|
||||||
|
}
|
||||||
|
assert all(report["checks"].values())
|
||||||
|
assert report["final_canonical_rows"] == expected_rows
|
||||||
|
assert report["checks"]["target_table_deltas_exact"] is True
|
||||||
|
assert report["cleanup"]["active_gateway_registry_clear"] is True
|
||||||
|
assert not os.path.lexists(runtime_root)
|
||||||
|
retained = json.loads(args.output.read_text(encoding="utf-8"))
|
||||||
|
assert retained["status"] == "pass"
|
||||||
|
assert args.output.stat().st_mode & 0o777 == 0o600
|
||||||
823
tests/test_run_leo_clone_lifecycle_checkpoint.py
Normal file
823
tests/test_run_leo_clone_lifecycle_checkpoint.py
Normal file
|
|
@ -0,0 +1,823 @@
|
||||||
|
"""Focused tests for the no-send clone lifecycle checkpoint."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import asyncio
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import uuid
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
REPO_ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
sys.path.insert(0, str(REPO_ROOT / "scripts"))
|
||||||
|
|
||||||
|
import apply_proposal as ap # noqa: E402
|
||||||
|
import run_leo_clone_lifecycle_checkpoint as lifecycle # noqa: E402
|
||||||
|
|
||||||
|
RUN_MARKER = "leo-lifecycle-unit-20260710"
|
||||||
|
CONVERSATION_MARKER = "conversation-memory-unit-20260710"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _clear_bound_registries() -> None:
|
||||||
|
lifecycle.bound._ACTIVE_TEMP_ROOTS.clear()
|
||||||
|
lifecycle.bound._ACTIVE_GATEWAY_CHILDREN.clear()
|
||||||
|
lifecycle.bound._LAST_TERMINATION_SIGNAL = None
|
||||||
|
yield
|
||||||
|
lifecycle.bound._ACTIVE_TEMP_ROOTS.clear()
|
||||||
|
lifecycle.bound._ACTIVE_GATEWAY_CHILDREN.clear()
|
||||||
|
lifecycle.bound._LAST_TERMINATION_SIGNAL = None
|
||||||
|
|
||||||
|
|
||||||
|
def _args(tmp_path: Path, **overrides: Any) -> argparse.Namespace:
|
||||||
|
review_secret = tmp_path / "kb-review.env"
|
||||||
|
apply_secret = tmp_path / "kb-apply.env"
|
||||||
|
review_secret.write_text("KB_REVIEW_PASSWORD=private\n", encoding="utf-8")
|
||||||
|
apply_secret.write_text("KB_APPLY_PASSWORD=private\n", encoding="utf-8")
|
||||||
|
values: dict[str, Any] = {
|
||||||
|
"container": "teleo-lifecycle-clone",
|
||||||
|
"db": "teleo",
|
||||||
|
"output": tmp_path / "receipt.json",
|
||||||
|
"run_marker": RUN_MARKER,
|
||||||
|
"conversation_marker": CONVERSATION_MARKER,
|
||||||
|
"chat_id": "clone-chat",
|
||||||
|
"user_id": "clone-user",
|
||||||
|
"user_name": "clone operator",
|
||||||
|
"turn_timeout": 30,
|
||||||
|
"review_secrets_file": str(review_secret),
|
||||||
|
"apply_secrets_file": str(apply_secret),
|
||||||
|
"copy_model_auth": True,
|
||||||
|
"operator_review": True,
|
||||||
|
"guarded_apply": True,
|
||||||
|
}
|
||||||
|
values.update(overrides)
|
||||||
|
return argparse.Namespace(**values)
|
||||||
|
|
||||||
|
|
||||||
|
def _proposal(fixture: dict[str, Any], state: str) -> dict[str, Any]:
|
||||||
|
row = {
|
||||||
|
"id": fixture["proposal_id"],
|
||||||
|
"proposal_type": "approve_claim",
|
||||||
|
"status": state,
|
||||||
|
"channel": "clone_lifecycle",
|
||||||
|
"source_ref": fixture["source_ref"],
|
||||||
|
"rationale": fixture["rationale"],
|
||||||
|
"payload": fixture["proposal_payload"],
|
||||||
|
"reviewed_by_handle": None,
|
||||||
|
"reviewed_by_agent_id": None,
|
||||||
|
"reviewed_at": None,
|
||||||
|
"review_note": None,
|
||||||
|
"applied_by_handle": None,
|
||||||
|
"applied_by_agent_id": None,
|
||||||
|
"applied_at": None,
|
||||||
|
}
|
||||||
|
if state in {"approved", "applied"}:
|
||||||
|
row.update(
|
||||||
|
{
|
||||||
|
"reviewed_by_handle": lifecycle.REVIEWER_HANDLE,
|
||||||
|
"reviewed_at": "2026-07-10T10:00:00+00:00",
|
||||||
|
"review_note": lifecycle.REVIEW_NOTE,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
if state == "applied":
|
||||||
|
row.update(
|
||||||
|
{
|
||||||
|
"applied_by_handle": lifecycle.APPLIED_BY_HANDLE,
|
||||||
|
"applied_by_agent_id": "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa",
|
||||||
|
"applied_at": "2026-07-10T10:01:00+00:00",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def _approval_snapshot(proposal: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"proposal_id": proposal["id"],
|
||||||
|
"proposal_type": proposal["proposal_type"],
|
||||||
|
"payload": proposal["payload"],
|
||||||
|
"reviewed_by_handle": proposal["reviewed_by_handle"],
|
||||||
|
"reviewed_by_agent_id": proposal["reviewed_by_agent_id"],
|
||||||
|
"reviewed_by_db_role": "kb_review",
|
||||||
|
"reviewed_at": proposal["reviewed_at"],
|
||||||
|
"review_note": proposal["review_note"],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_fixture_is_deterministic_narrow_and_accepted_by_guarded_apply() -> None:
|
||||||
|
first = lifecycle.build_lifecycle_fixture(RUN_MARKER)
|
||||||
|
second = lifecycle.build_lifecycle_fixture(RUN_MARKER)
|
||||||
|
|
||||||
|
assert first == second
|
||||||
|
assert len(first["apply_payload"]["claims"]) == 1
|
||||||
|
assert len(first["apply_payload"]["sources"]) == 1
|
||||||
|
assert len(first["apply_payload"]["evidence"]) == 1
|
||||||
|
assert first["apply_payload"]["edges"] == []
|
||||||
|
assert first["apply_payload"]["reasoning_tools"] == []
|
||||||
|
assert first["proposal_payload"]["lifecycle_checkpoint"]["model_authority"] == "stage_only"
|
||||||
|
for key in ("proposal_id", "claim_id", "source_id"):
|
||||||
|
uuid.UUID(first[key])
|
||||||
|
|
||||||
|
approved = _proposal(first, "approved")
|
||||||
|
sql = ap.build_apply_sql(approved, lifecycle.APPLIED_BY_HANDLE)
|
||||||
|
assert "kb_stage.assert_approved_proposal" in sql
|
||||||
|
assert "kb_stage.finish_approved_proposal" in sql
|
||||||
|
assert first["claim_id"] in sql
|
||||||
|
assert first["source_id"] in sql
|
||||||
|
|
||||||
|
|
||||||
|
def test_stage_sql_is_idempotent_and_cannot_write_canonical_tables() -> None:
|
||||||
|
fixture = lifecycle.build_lifecycle_fixture(RUN_MARKER)
|
||||||
|
sql = lifecycle.build_stage_sql(fixture)
|
||||||
|
lowered = sql.lower()
|
||||||
|
|
||||||
|
assert "insert into kb_stage.kb_proposals" in lowered
|
||||||
|
assert "on conflict (id) do nothing" in lowered
|
||||||
|
assert "pending_review" in sql
|
||||||
|
assert fixture["source_ref"] in sql
|
||||||
|
assert "lifecycle stage validation failed" in lowered
|
||||||
|
assert "marker_count <> 1 or exact_count <> 1" in lowered
|
||||||
|
assert lowered.index("raise exception") < lowered.index("commit;")
|
||||||
|
assert "insert into public." not in lowered
|
||||||
|
assert "update public." not in lowered
|
||||||
|
assert "delete from public." not in lowered
|
||||||
|
|
||||||
|
|
||||||
|
def test_unrelated_row_guard_is_read_only_and_excludes_only_deterministic_rows(
|
||||||
|
monkeypatch: pytest.MonkeyPatch,
|
||||||
|
) -> None:
|
||||||
|
fixture = lifecycle.build_lifecycle_fixture(RUN_MARKER)
|
||||||
|
captured: dict[str, str] = {}
|
||||||
|
|
||||||
|
def fake_psql(_container: str, _database: str, sql: str) -> dict[str, Any]:
|
||||||
|
captured["sql"] = sql
|
||||||
|
return {table: {"count": 1, "rowset_md5": "same"} for table in lifecycle.LIFECYCLE_TABLES}
|
||||||
|
|
||||||
|
monkeypatch.setattr(lifecycle.bound, "_psql_json", fake_psql)
|
||||||
|
|
||||||
|
snapshot = lifecycle.target_unrelated_guard_snapshot("clone", "teleo", fixture)
|
||||||
|
sql = captured["sql"].lower()
|
||||||
|
|
||||||
|
assert set(snapshot) == set(lifecycle.LIFECYCLE_TABLES)
|
||||||
|
assert "begin transaction read only" in sql
|
||||||
|
assert fixture["proposal_id"] in sql
|
||||||
|
assert fixture["claim_id"] in sql
|
||||||
|
assert fixture["source_id"] in sql
|
||||||
|
assert lifecycle.APPROVAL_TABLE in sql
|
||||||
|
assert "insert into" not in sql
|
||||||
|
assert "update " not in sql
|
||||||
|
assert "delete from" not in sql
|
||||||
|
|
||||||
|
|
||||||
|
def test_stage_replay_returns_the_same_pending_proposal(
|
||||||
|
monkeypatch: pytest.MonkeyPatch,
|
||||||
|
) -> None:
|
||||||
|
fixture = lifecycle.build_lifecycle_fixture(RUN_MARKER)
|
||||||
|
pending = _proposal(fixture, "pending_review")
|
||||||
|
monkeypatch.setattr(
|
||||||
|
lifecycle,
|
||||||
|
"assert_disposable_target",
|
||||||
|
lambda _container, _database: ({"id": "clone"}, {"id": "production"}),
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(
|
||||||
|
lifecycle.bound,
|
||||||
|
"_psql_json",
|
||||||
|
lambda _container, _database, _sql: {
|
||||||
|
"created": False,
|
||||||
|
"marker_match_count": 1,
|
||||||
|
"proposal": pending,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
replay = lifecycle.stage_lifecycle_proposal("clone", "teleo", fixture)
|
||||||
|
|
||||||
|
assert replay["created"] is False
|
||||||
|
assert replay["idempotent_existing"] is True
|
||||||
|
assert replay["proposal"] == pending
|
||||||
|
assert replay["canonical_row_ids"]["public.claims"] == [fixture["claim_id"]]
|
||||||
|
|
||||||
|
|
||||||
|
def test_staging_wrapper_hard_binds_target_and_exposes_no_review_apply_or_send(tmp_path: Path) -> None:
|
||||||
|
wrapper = lifecycle.build_lifecycle_wrapper(
|
||||||
|
kb_tool=tmp_path / "kb_tool.py",
|
||||||
|
tool_log=tmp_path / "tool.jsonl",
|
||||||
|
container="clone-container",
|
||||||
|
database="teleo_clone",
|
||||||
|
run_marker=RUN_MARKER,
|
||||||
|
run_nonce="unit-test-nonce",
|
||||||
|
)
|
||||||
|
|
||||||
|
assert "TARGET_CONTAINER=clone-container" in wrapper
|
||||||
|
assert "TARGET_DATABASE=teleo_clone" in wrapper
|
||||||
|
assert wrapper.startswith("#!/bin/bash\n")
|
||||||
|
assert f"DOCKER={lifecycle.bound.SYSTEM_DOCKER}" in wrapper
|
||||||
|
assert f"export PATH={lifecycle.bound.SYSTEM_EXEC_PATH}" in wrapper
|
||||||
|
assert "/usr/bin/env bash" not in wrapper
|
||||||
|
assert "$(docker " not in wrapper
|
||||||
|
assert lifecycle.INTERNAL_STAGE_COMMAND in wrapper
|
||||||
|
assert f'${{1:-}}" == {lifecycle.STAGE_COMMAND}' in wrapper
|
||||||
|
assert "accepts no model-supplied arguments" in wrapper
|
||||||
|
assert "approve_proposal.py" not in wrapper
|
||||||
|
assert "apply_proposal.py" not in wrapper
|
||||||
|
assert "send_message" not in wrapper
|
||||||
|
|
||||||
|
|
||||||
|
def test_lifecycle_surface_temporarily_adds_only_the_stage_verb(
|
||||||
|
monkeypatch: pytest.MonkeyPatch,
|
||||||
|
tmp_path: Path,
|
||||||
|
) -> None:
|
||||||
|
original_commands = lifecycle.bound.READ_ONLY_BRIDGE_COMMANDS
|
||||||
|
|
||||||
|
def fake_install(_profile: Path) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"allowed_tools": ["skills_list", "skill_view", "terminal"],
|
||||||
|
"read_only_bridge_commands": sorted(lifecycle.bound.READ_ONLY_BRIDGE_COMMANDS),
|
||||||
|
"send_message_tool_enabled": False,
|
||||||
|
}
|
||||||
|
|
||||||
|
monkeypatch.setattr(lifecycle.bound, "install_checkpoint_tool_surface", fake_install)
|
||||||
|
|
||||||
|
with lifecycle.lifecycle_gateway_surface(allow_staging=True):
|
||||||
|
assert lifecycle.STAGE_COMMAND in lifecycle.bound.READ_ONLY_BRIDGE_COMMANDS
|
||||||
|
surface = lifecycle.bound.install_checkpoint_tool_surface(tmp_path)
|
||||||
|
assert surface["staging_bridge_commands"] == [lifecycle.STAGE_COMMAND]
|
||||||
|
assert surface["model_can_stage_pending_proposal"] is True
|
||||||
|
assert surface["model_can_approve_or_apply"] is False
|
||||||
|
assert surface["operator_review_apply_tools_exposed"] is False
|
||||||
|
assert "send_message" not in surface["allowed_tools"]
|
||||||
|
|
||||||
|
assert original_commands == lifecycle.bound.READ_ONLY_BRIDGE_COMMANDS
|
||||||
|
assert lifecycle.STAGE_COMMAND not in lifecycle.bound.READ_ONLY_BRIDGE_COMMANDS
|
||||||
|
|
||||||
|
with lifecycle.lifecycle_gateway_surface(allow_staging=False):
|
||||||
|
surface = lifecycle.bound.install_checkpoint_tool_surface(tmp_path)
|
||||||
|
assert surface["staging_bridge_commands"] == []
|
||||||
|
assert surface["model_can_stage_pending_proposal"] is False
|
||||||
|
with pytest.raises(lifecycle.bound.CheckpointError):
|
||||||
|
lifecycle.bound.validate_read_only_bridge_argv([lifecycle.STAGE_COMMAND])
|
||||||
|
|
||||||
|
|
||||||
|
def test_prompts_keep_conversation_marker_out_of_mutation_and_recall_prompt() -> None:
|
||||||
|
fixture = lifecycle.build_lifecycle_fixture(RUN_MARKER)
|
||||||
|
prompts = lifecycle.lifecycle_prompts(fixture, CONVERSATION_MARKER)
|
||||||
|
|
||||||
|
assert CONVERSATION_MARKER in prompts["T1_conversation_marker"]
|
||||||
|
assert CONVERSATION_MARKER not in prompts["T2_stage_pending_proposal"]
|
||||||
|
assert CONVERSATION_MARKER not in prompts["T5_recall_and_applied_readback"]
|
||||||
|
assert CONVERSATION_MARKER not in json.dumps(fixture, sort_keys=True)
|
||||||
|
assert RUN_MARKER in prompts["T2_stage_pending_proposal"]
|
||||||
|
assert RUN_MARKER not in prompts["T5_recall_and_applied_readback"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_lifecycle_state_receipt_is_exact_and_rejects_extra_or_contradictory_state() -> None:
|
||||||
|
expected = {
|
||||||
|
"phase": "T2",
|
||||||
|
"proposal_id": "537c3087-b795-5382-8335-e0928d1ac96c",
|
||||||
|
"status": "pending_review",
|
||||||
|
"canonical": False,
|
||||||
|
}
|
||||||
|
reply = "Proposal staged and not canonical.\nLIFECYCLE_STATE: " + json.dumps(expected)
|
||||||
|
|
||||||
|
assert lifecycle.parse_lifecycle_state(reply) == {"valid": True, "value": expected}
|
||||||
|
assert lifecycle.lifecycle_state_matches(reply, expected) is True
|
||||||
|
assert lifecycle.lifecycle_state_matches(reply.replace("}", ', "unverified": true}', 1), expected) is False
|
||||||
|
assert lifecycle._reply_has_phase_contradiction("T2", reply) is False
|
||||||
|
assert (
|
||||||
|
lifecycle._reply_has_phase_contradiction("T2", "The proposal is now canonical.\n" + reply.splitlines()[-1])
|
||||||
|
is True
|
||||||
|
)
|
||||||
|
assert lifecycle.parse_lifecycle_state("LIFECYCLE_STATE: not-json")["valid"] is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_phase_row_read_requires_the_exact_full_uuid() -> None:
|
||||||
|
turn = {"tool_invocations": [{"argv": ["show-proposal", "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa"]}]}
|
||||||
|
|
||||||
|
assert lifecycle._phase_has_bound_read(turn, "show-proposal", "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa") is True
|
||||||
|
assert lifecycle._phase_has_bound_read(turn, "show-proposal", "bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb") is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_validation_requires_explicit_clone_review_apply_and_auth(tmp_path: Path) -> None:
|
||||||
|
lifecycle.validate_args(_args(tmp_path))
|
||||||
|
|
||||||
|
with pytest.raises(lifecycle.bound.CheckpointError, match="production"):
|
||||||
|
lifecycle.validate_args(_args(tmp_path, container=lifecycle.bound.PRODUCTION_CONTAINER))
|
||||||
|
with pytest.raises(lifecycle.bound.CheckpointError, match="operator-review"):
|
||||||
|
lifecycle.validate_args(_args(tmp_path, operator_review=False))
|
||||||
|
with pytest.raises(lifecycle.bound.CheckpointError, match="guarded-apply"):
|
||||||
|
lifecycle.validate_args(_args(tmp_path, guarded_apply=False))
|
||||||
|
with pytest.raises(lifecycle.bound.CheckpointError, match="copy-model-auth"):
|
||||||
|
lifecycle.validate_args(_args(tmp_path, copy_model_auth=False))
|
||||||
|
|
||||||
|
|
||||||
|
def test_validation_rejects_receipt_inside_live_profile(
|
||||||
|
monkeypatch: pytest.MonkeyPatch,
|
||||||
|
tmp_path: Path,
|
||||||
|
) -> None:
|
||||||
|
live = tmp_path / "live-profile"
|
||||||
|
live.mkdir()
|
||||||
|
monkeypatch.setattr(lifecycle.bound, "LIVE_PROFILE", live)
|
||||||
|
|
||||||
|
with pytest.raises(lifecycle.bound.CheckpointError, match="outside the live leoclean profile"):
|
||||||
|
lifecycle.validate_args(_args(tmp_path, output=live / "receipt.json"))
|
||||||
|
|
||||||
|
|
||||||
|
def test_disposable_identity_rejects_an_alias_of_production(
|
||||||
|
monkeypatch: pytest.MonkeyPatch,
|
||||||
|
) -> None:
|
||||||
|
monkeypatch.setattr(
|
||||||
|
lifecycle.bound,
|
||||||
|
"container_identity",
|
||||||
|
lambda name: {"id": "same-id", "name": name, "image": "postgres"},
|
||||||
|
)
|
||||||
|
|
||||||
|
with pytest.raises(lifecycle.bound.CheckpointError, match="production container identity"):
|
||||||
|
lifecycle.assert_disposable_target("alias-container", "teleo")
|
||||||
|
|
||||||
|
|
||||||
|
def _install_fake_runtime(
|
||||||
|
monkeypatch: pytest.MonkeyPatch,
|
||||||
|
tmp_path: Path,
|
||||||
|
args: argparse.Namespace,
|
||||||
|
) -> tuple[dict[str, str], dict[str, Any]]:
|
||||||
|
fixture = lifecycle.build_lifecycle_fixture(args.run_marker)
|
||||||
|
expected_rows = lifecycle.guarded._expected_bundle_rows(fixture["apply_payload"])
|
||||||
|
empty_rows = {key: [] for key in expected_rows}
|
||||||
|
base_counts = {
|
||||||
|
"public.claims": 100,
|
||||||
|
"public.sources": 200,
|
||||||
|
"public.claim_evidence": 300,
|
||||||
|
"public.claim_edges": 400,
|
||||||
|
"public.reasoning_tools": 5,
|
||||||
|
"kb_stage.kb_proposals": 9,
|
||||||
|
lifecycle.APPROVAL_TABLE: 4,
|
||||||
|
}
|
||||||
|
stage_counts = dict(base_counts)
|
||||||
|
stage_counts["kb_stage.kb_proposals"] += 1
|
||||||
|
approved_counts = dict(stage_counts)
|
||||||
|
approved_counts[lifecycle.APPROVAL_TABLE] += 1
|
||||||
|
final_counts = dict(approved_counts)
|
||||||
|
final_counts["public.claims"] += 1
|
||||||
|
final_counts["public.sources"] += 1
|
||||||
|
final_counts["public.claim_evidence"] += 1
|
||||||
|
state: dict[str, Any] = {"phase": "initial", "invocations": []}
|
||||||
|
base_snapshot = {
|
||||||
|
"counts": base_counts,
|
||||||
|
"rowset_md5": {table: f"production-{index}" for index, table in enumerate(base_counts)},
|
||||||
|
}
|
||||||
|
target_identity = {
|
||||||
|
"id": "clone-id",
|
||||||
|
"name": args.container,
|
||||||
|
"image": "postgres",
|
||||||
|
"labels": {
|
||||||
|
lifecycle.bound.DISPOSABLE_LABEL_KEY: lifecycle.bound.DISPOSABLE_LABEL_VALUE,
|
||||||
|
},
|
||||||
|
"network_mode": "none",
|
||||||
|
"ports": {},
|
||||||
|
"mount_sources": ["/clone-data"],
|
||||||
|
"started_at": "clone-start",
|
||||||
|
}
|
||||||
|
production_identity = {
|
||||||
|
"id": "production-id",
|
||||||
|
"name": lifecycle.bound.PRODUCTION_CONTAINER,
|
||||||
|
"image": "postgres",
|
||||||
|
"labels": {},
|
||||||
|
"network_mode": "bridge",
|
||||||
|
"ports": {},
|
||||||
|
"mount_sources": ["/production-data"],
|
||||||
|
"started_at": "production-start",
|
||||||
|
}
|
||||||
|
|
||||||
|
monkeypatch.setattr(
|
||||||
|
lifecycle,
|
||||||
|
"assert_disposable_target",
|
||||||
|
lambda _container, _database: (
|
||||||
|
dict(target_identity),
|
||||||
|
dict(production_identity),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(
|
||||||
|
lifecycle.bound,
|
||||||
|
"container_identity",
|
||||||
|
lambda container: dict(
|
||||||
|
production_identity if container == lifecycle.bound.PRODUCTION_CONTAINER else target_identity
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
def fake_guard_snapshot(container: str, _database: str, **_kwargs: Any) -> dict[str, Any]:
|
||||||
|
if container == lifecycle.bound.PRODUCTION_CONTAINER:
|
||||||
|
snapshot = json.loads(json.dumps(base_snapshot))
|
||||||
|
snapshot["database_identity"] = {
|
||||||
|
"current_database": args.db,
|
||||||
|
"system_identifier": "production-system",
|
||||||
|
"transaction_read_only": "off",
|
||||||
|
}
|
||||||
|
return snapshot
|
||||||
|
identity = {
|
||||||
|
"current_database": args.db,
|
||||||
|
"system_identifier": "clone-system",
|
||||||
|
"transaction_read_only": "off",
|
||||||
|
}
|
||||||
|
if state["phase"] == "applied":
|
||||||
|
snapshot = json.loads(json.dumps(base_snapshot))
|
||||||
|
snapshot["counts"] = dict(final_counts)
|
||||||
|
for table in (
|
||||||
|
"public.claims",
|
||||||
|
"public.sources",
|
||||||
|
"public.claim_evidence",
|
||||||
|
"kb_stage.kb_proposals",
|
||||||
|
lifecycle.APPROVAL_TABLE,
|
||||||
|
):
|
||||||
|
snapshot["rowset_md5"][table] = "changed"
|
||||||
|
snapshot["database_identity"] = identity
|
||||||
|
return snapshot
|
||||||
|
snapshot = json.loads(json.dumps(base_snapshot))
|
||||||
|
snapshot["database_identity"] = identity
|
||||||
|
return snapshot
|
||||||
|
|
||||||
|
monkeypatch.setattr(lifecycle.bound, "database_guard_snapshot", fake_guard_snapshot)
|
||||||
|
monkeypatch.setattr(
|
||||||
|
lifecycle.bound,
|
||||||
|
"service_state",
|
||||||
|
lambda: {"returncode": 0, "ActiveState": "active", "MainPID": "123", "NRestarts": "0"},
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(
|
||||||
|
lifecycle.bound,
|
||||||
|
"bridge_hashes",
|
||||||
|
lambda: {"wrapper": "live-wrapper-hash", "bridge_skill": "live-skill-hash"},
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(
|
||||||
|
lifecycle,
|
||||||
|
"target_unrelated_guard_snapshot",
|
||||||
|
lambda _container, _database, _fixture: {
|
||||||
|
table: {"count": base_counts[table], "rowset_md5": f"unrelated-{table}"}
|
||||||
|
for table in lifecycle.LIFECYCLE_TABLES
|
||||||
|
},
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(lifecycle, "_source_manifest", lambda: [{"repo_relative_path": "stable", "sha256": "x"}])
|
||||||
|
|
||||||
|
def fake_proposal_readback(_container: str, _database: str, _proposal_id: str) -> dict[str, Any] | None:
|
||||||
|
if state["phase"] == "initial":
|
||||||
|
return None
|
||||||
|
return _proposal(fixture, state["phase"])
|
||||||
|
|
||||||
|
monkeypatch.setattr(lifecycle.guarded, "_proposal_readback", fake_proposal_readback)
|
||||||
|
monkeypatch.setattr(
|
||||||
|
lifecycle.guarded,
|
||||||
|
"_exact_bundle_readback",
|
||||||
|
lambda _container, _database, _payload: expected_rows if state["phase"] == "applied" else empty_rows,
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(
|
||||||
|
lifecycle.guarded,
|
||||||
|
"_approval_snapshot_readback",
|
||||||
|
lambda _container, _database, _proposal_id: _approval_snapshot(_proposal(fixture, "approved")),
|
||||||
|
)
|
||||||
|
|
||||||
|
runtime_root = tmp_path / "private-runtime"
|
||||||
|
runtime_profile = runtime_root / "profile"
|
||||||
|
|
||||||
|
def fake_copy_profile(**_kwargs: Any) -> tuple[Path, dict[str, Any]]:
|
||||||
|
runtime_profile.mkdir(parents=True)
|
||||||
|
lifecycle.bound.register_temp_root(runtime_root)
|
||||||
|
return runtime_profile, {"passes": True, "secret_contents_read_or_recorded": False}
|
||||||
|
|
||||||
|
monkeypatch.setattr(lifecycle.bound, "copy_profile", fake_copy_profile)
|
||||||
|
monkeypatch.setattr(
|
||||||
|
lifecycle.bound,
|
||||||
|
"copy_ephemeral_model_auth",
|
||||||
|
lambda _profile: {
|
||||||
|
"mode": "ephemeral_file_copy",
|
||||||
|
"source_contents_recorded": False,
|
||||||
|
"source_fingerprint_recorded": False,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(
|
||||||
|
lifecycle.bound,
|
||||||
|
"load_ephemeral_provider_environment",
|
||||||
|
lambda _profile: ({}, {"bound_credential_count": 0, "credential_contents_recorded": False}),
|
||||||
|
)
|
||||||
|
tool_log = runtime_profile / "checkpoint-tool-calls.jsonl"
|
||||||
|
monkeypatch.setattr(
|
||||||
|
lifecycle,
|
||||||
|
"patch_lifecycle_bridge",
|
||||||
|
lambda _profile, **_kwargs: {
|
||||||
|
"wrapper_path": str(runtime_profile / "bin" / "teleo-kb"),
|
||||||
|
"bridge_skill_path": str(runtime_profile / "skills" / "teleo-kb-bridge" / "SKILL.md"),
|
||||||
|
"tool_log_path": str(tool_log),
|
||||||
|
"run_nonce": "lifecycle-test-nonce",
|
||||||
|
"model_can_approve": False,
|
||||||
|
"model_can_apply": False,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
async def fake_turn(
|
||||||
|
_args: argparse.Namespace,
|
||||||
|
_profile: Path,
|
||||||
|
*,
|
||||||
|
phase: str,
|
||||||
|
prompt: str,
|
||||||
|
provider_environment: dict[str, str],
|
||||||
|
tool_log: Path,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
assert provider_environment == {}
|
||||||
|
assert tool_log == runtime_profile / "checkpoint-tool-calls.jsonl"
|
||||||
|
command_argvs: list[list[str]]
|
||||||
|
if phase == "T1_conversation_marker":
|
||||||
|
reply = (
|
||||||
|
f"I remember {args.conversation_marker}; conversation memory is not canonical Postgres.\n"
|
||||||
|
f'{lifecycle.LIFECYCLE_STATE_PREFIX} {{"phase":"T1","conversation_marker":'
|
||||||
|
f'"{args.conversation_marker}","kb_mutated":false}}'
|
||||||
|
)
|
||||||
|
command_argvs = []
|
||||||
|
elif phase == "T2_stage_pending_proposal":
|
||||||
|
state["phase"] = "pending_review"
|
||||||
|
reply = (
|
||||||
|
f"Staged proposal {fixture['proposal_id']} as pending_review, not canonical.\n"
|
||||||
|
f'{lifecycle.LIFECYCLE_STATE_PREFIX} {{"phase":"T2","proposal_id":"{fixture["proposal_id"]}",'
|
||||||
|
'"status":"pending_review","canonical":false}'
|
||||||
|
)
|
||||||
|
command_argvs = [
|
||||||
|
[lifecycle.STAGE_COMMAND],
|
||||||
|
["show-proposal", fixture["proposal_id"]],
|
||||||
|
]
|
||||||
|
elif phase == "T3_approved_unapplied_readback":
|
||||||
|
reply = (
|
||||||
|
f"Proposal {fixture['proposal_id']} is approved, applied_at NULL, and not canonical.\n"
|
||||||
|
f'{lifecycle.LIFECYCLE_STATE_PREFIX} {{"phase":"T3","proposal_id":"{fixture["proposal_id"]}",'
|
||||||
|
'"status":"approved","applied":false,"canonical":false}'
|
||||||
|
)
|
||||||
|
command_argvs = [["show-proposal", fixture["proposal_id"]]]
|
||||||
|
else:
|
||||||
|
reply = (
|
||||||
|
f"Marker {args.conversation_marker}; proposal {fixture['proposal_id']} is applied; "
|
||||||
|
f"canonical claim {fixture['claim_id']} and source {fixture['source_id']}.\n"
|
||||||
|
f'{lifecycle.LIFECYCLE_STATE_PREFIX} {{"phase":"T5","conversation_marker":'
|
||||||
|
f'"{args.conversation_marker}","proposal_id":"{fixture["proposal_id"]}","status":"applied",'
|
||||||
|
f'"claim_id":"{fixture["claim_id"]}","source_id":"{fixture["source_id"]}"}}'
|
||||||
|
)
|
||||||
|
command_argvs = [
|
||||||
|
["show-proposal", fixture["proposal_id"]],
|
||||||
|
["show", fixture["claim_id"]],
|
||||||
|
]
|
||||||
|
invocations = [
|
||||||
|
{
|
||||||
|
"invocation_id": f"{phase}-{index}",
|
||||||
|
"container": args.bound_container_id,
|
||||||
|
"database": args.db,
|
||||||
|
"argv": command_argv,
|
||||||
|
"returncode": 0,
|
||||||
|
}
|
||||||
|
for index, command_argv in enumerate(command_argvs)
|
||||||
|
]
|
||||||
|
state["invocations"].extend(invocations)
|
||||||
|
pid = {
|
||||||
|
"T1_conversation_marker": 101,
|
||||||
|
"T2_stage_pending_proposal": 102,
|
||||||
|
"T3_approved_unapplied_readback": 103,
|
||||||
|
"T5_recall_and_applied_readback": 105,
|
||||||
|
}[phase]
|
||||||
|
gateway = {
|
||||||
|
"authorized": True,
|
||||||
|
"handler_invoked": True,
|
||||||
|
"posted_to_telegram": False,
|
||||||
|
"model_free_fallback_used": False,
|
||||||
|
"reply": reply,
|
||||||
|
"session_key": "telegram:clone-chat",
|
||||||
|
"transcript_tool_trace": {
|
||||||
|
"session_id": "persisted-private-session",
|
||||||
|
"events": [
|
||||||
|
event
|
||||||
|
for item in invocations
|
||||||
|
for event in (
|
||||||
|
{"phase": "call", "tool_name": "terminal", "tool_call_id": item["invocation_id"]},
|
||||||
|
{
|
||||||
|
"phase": "result",
|
||||||
|
"tool_name": "terminal",
|
||||||
|
"tool_call_id": item["invocation_id"],
|
||||||
|
"content": '{"exit_code": 0}',
|
||||||
|
},
|
||||||
|
)
|
||||||
|
],
|
||||||
|
},
|
||||||
|
"tool_surface": {
|
||||||
|
"allowed_tools": ["skills_list", "skill_view", "terminal"],
|
||||||
|
"actual_registry_tools": ["skills_list", "skill_view", "terminal"],
|
||||||
|
"send_message_tool_enabled": False,
|
||||||
|
"gateway_adapters_verified_mapping": True,
|
||||||
|
"gateway_adapter_count": 0,
|
||||||
|
"model_can_approve_or_apply": False,
|
||||||
|
"operator_review_apply_tools_exposed": False,
|
||||||
|
},
|
||||||
|
"child_process": {
|
||||||
|
"pid": pid,
|
||||||
|
"readiness_verified": True,
|
||||||
|
"timed_out": False,
|
||||||
|
"exitcode": 0,
|
||||||
|
"termination": {"attempted": False},
|
||||||
|
"alive_after_readback": False,
|
||||||
|
"process_group_alive_after_readback": False,
|
||||||
|
"result_transport": "private_temp_file",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
"phase": phase,
|
||||||
|
"prompt": prompt,
|
||||||
|
"prompt_sha256": "prompt-hash",
|
||||||
|
"gateway": gateway,
|
||||||
|
"tool_invocations": invocations,
|
||||||
|
}
|
||||||
|
|
||||||
|
monkeypatch.setattr(lifecycle, "run_gateway_turn", fake_turn)
|
||||||
|
monkeypatch.setattr(
|
||||||
|
lifecycle,
|
||||||
|
"stage_lifecycle_proposal",
|
||||||
|
lambda _container, _database, _fixture: {
|
||||||
|
"created": False,
|
||||||
|
"idempotent_existing": True,
|
||||||
|
"marker_match_count": 1,
|
||||||
|
"proposal": _proposal(fixture, "pending_review"),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
def fake_review(
|
||||||
|
_args: argparse.Namespace, _proposal_id: str
|
||||||
|
) -> tuple[subprocess.CompletedProcess[str], subprocess.CompletedProcess[str]]:
|
||||||
|
dry = subprocess.CompletedProcess([], 0, stdout="select kb_stage.approve_strict_proposal(...);", stderr="")
|
||||||
|
state["phase"] = "approved"
|
||||||
|
applied = subprocess.CompletedProcess([], 0, stdout="approved", stderr="")
|
||||||
|
return dry, applied
|
||||||
|
|
||||||
|
def fake_apply(
|
||||||
|
_args: argparse.Namespace, _proposal_id: str
|
||||||
|
) -> tuple[subprocess.CompletedProcess[str], subprocess.CompletedProcess[str]]:
|
||||||
|
dry = subprocess.CompletedProcess(
|
||||||
|
[],
|
||||||
|
0,
|
||||||
|
stdout=("select kb_stage.assert_approved_proposal(...); select kb_stage.finish_approved_proposal(...);"),
|
||||||
|
stderr="",
|
||||||
|
)
|
||||||
|
state["phase"] = "applied"
|
||||||
|
applied = subprocess.CompletedProcess([], 0, stdout="applied", stderr="")
|
||||||
|
return dry, applied
|
||||||
|
|
||||||
|
monkeypatch.setattr(lifecycle, "run_operator_review", fake_review)
|
||||||
|
monkeypatch.setattr(lifecycle, "run_guarded_apply", fake_apply)
|
||||||
|
|
||||||
|
def fake_counts(_container: str, _database: str, **_kwargs: Any) -> dict[str, int]:
|
||||||
|
if state["phase"] == "applied":
|
||||||
|
return dict(final_counts)
|
||||||
|
if state["phase"] == "approved":
|
||||||
|
return dict(approved_counts)
|
||||||
|
return dict(stage_counts)
|
||||||
|
|
||||||
|
monkeypatch.setattr(lifecycle.bound, "database_counts", fake_counts)
|
||||||
|
monkeypatch.setattr(
|
||||||
|
lifecycle.bound,
|
||||||
|
"read_tool_proof",
|
||||||
|
lambda _path, _container, _database, **_kwargs: {
|
||||||
|
"event_count": len(state["invocations"]) * 2,
|
||||||
|
"parse_errors": [],
|
||||||
|
"duplicate_invocation_ids": [],
|
||||||
|
"complete_start_end_pairing": True,
|
||||||
|
"invocations": list(state["invocations"]),
|
||||||
|
"invocation_count": len(state["invocations"]),
|
||||||
|
"all_bound_to_supplied_target": bool(state["invocations"]),
|
||||||
|
"database_read_only_required": False,
|
||||||
|
"all_completed_successfully": bool(state["invocations"]),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
return state, {"runtime_root": runtime_root, "expected_rows": expected_rows}
|
||||||
|
|
||||||
|
|
||||||
|
def test_mocked_full_lifecycle_produces_private_passing_receipt_and_cleanup(
|
||||||
|
monkeypatch: pytest.MonkeyPatch,
|
||||||
|
tmp_path: Path,
|
||||||
|
) -> None:
|
||||||
|
args = _args(tmp_path)
|
||||||
|
_state, proof = _install_fake_runtime(monkeypatch, tmp_path, args)
|
||||||
|
|
||||||
|
report = asyncio.run(lifecycle.run_lifecycle(args))
|
||||||
|
|
||||||
|
assert report["status"] == "pass", {
|
||||||
|
"errors": report["errors"],
|
||||||
|
"failed_checks": [name for name, passed in report["checks"].items() if not passed],
|
||||||
|
}
|
||||||
|
assert all(report["checks"].values())
|
||||||
|
assert report["final_canonical_rows"] == proof["expected_rows"]
|
||||||
|
assert report["target_deltas"]["baseline_to_final"] == {
|
||||||
|
"kb_stage.kb_proposals": 1,
|
||||||
|
lifecycle.APPROVAL_TABLE: 1,
|
||||||
|
"public.claim_edges": 0,
|
||||||
|
"public.claim_evidence": 1,
|
||||||
|
"public.claims": 1,
|
||||||
|
"public.reasoning_tools": 0,
|
||||||
|
"public.sources": 1,
|
||||||
|
}
|
||||||
|
assert report["production_invariants"]["guard_snapshot_unchanged"] is True
|
||||||
|
assert report["isolated_restart"]["same_persisted_session_id"] is True
|
||||||
|
assert report["cleanup"]["active_gateway_registry_clear"] is True
|
||||||
|
assert not os.path.lexists(proof["runtime_root"])
|
||||||
|
retained = json.loads(args.output.read_text(encoding="utf-8"))
|
||||||
|
assert retained["status"] == "pass"
|
||||||
|
assert args.output.stat().st_mode & 0o777 == 0o600
|
||||||
|
|
||||||
|
|
||||||
|
def test_mocked_lifecycle_fails_if_target_name_is_rebound_after_execution(
|
||||||
|
monkeypatch: pytest.MonkeyPatch,
|
||||||
|
tmp_path: Path,
|
||||||
|
) -> None:
|
||||||
|
args = _args(tmp_path)
|
||||||
|
_install_fake_runtime(monkeypatch, tmp_path, args)
|
||||||
|
calls = 0
|
||||||
|
|
||||||
|
def replaced_identity(_container: str) -> dict[str, Any]:
|
||||||
|
nonlocal calls
|
||||||
|
calls += 1
|
||||||
|
return {
|
||||||
|
"id": "replacement-id",
|
||||||
|
"name": args.container,
|
||||||
|
"image": "postgres",
|
||||||
|
"labels": {
|
||||||
|
lifecycle.bound.DISPOSABLE_LABEL_KEY: lifecycle.bound.DISPOSABLE_LABEL_VALUE,
|
||||||
|
},
|
||||||
|
"network_mode": "none",
|
||||||
|
"ports": {},
|
||||||
|
"mount_sources": ["/replacement-data"],
|
||||||
|
"started_at": "replacement-start",
|
||||||
|
}
|
||||||
|
|
||||||
|
monkeypatch.setattr(lifecycle.bound, "container_identity", replaced_identity)
|
||||||
|
report = asyncio.run(lifecycle.run_lifecycle(args))
|
||||||
|
|
||||||
|
assert calls == 1
|
||||||
|
assert report["status"] == "fail"
|
||||||
|
assert any("pinned container identity" in item["message"] for item in report["errors"])
|
||||||
|
assert report["checks"]["target_container_name_still_resolves_to_pinned_identity"] is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_mocked_lifecycle_fails_if_t1_changes_database_before_authorized_stage(
|
||||||
|
monkeypatch: pytest.MonkeyPatch,
|
||||||
|
tmp_path: Path,
|
||||||
|
) -> None:
|
||||||
|
args = _args(tmp_path)
|
||||||
|
_install_fake_runtime(monkeypatch, tmp_path, args)
|
||||||
|
original_snapshot = lifecycle.bound.database_guard_snapshot
|
||||||
|
clone_reads = 0
|
||||||
|
|
||||||
|
def mutate_after_t1(container: str, database: str, **kwargs: Any) -> dict[str, Any]:
|
||||||
|
nonlocal clone_reads
|
||||||
|
snapshot = original_snapshot(container, database, **kwargs)
|
||||||
|
if container != lifecycle.bound.PRODUCTION_CONTAINER:
|
||||||
|
clone_reads += 1
|
||||||
|
if clone_reads == 2:
|
||||||
|
snapshot = json.loads(json.dumps(snapshot))
|
||||||
|
snapshot["rowset_md5"]["kb_stage.kb_proposals"] = "unexpected-t1-write"
|
||||||
|
return snapshot
|
||||||
|
|
||||||
|
monkeypatch.setattr(lifecycle.bound, "database_guard_snapshot", mutate_after_t1)
|
||||||
|
report = asyncio.run(lifecycle.run_lifecycle(args))
|
||||||
|
|
||||||
|
assert report["status"] == "fail"
|
||||||
|
assert any("T1 changed clone database state" in item["message"] for item in report["errors"])
|
||||||
|
assert report["checks"]["T1_database_unchanged_and_staging_absent"] is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_source_contains_no_direct_canonical_write_service_restart_or_delivery_adapter() -> None:
|
||||||
|
source = (REPO_ROOT / "scripts" / "run_leo_clone_lifecycle_checkpoint.py").read_text(encoding="utf-8").lower()
|
||||||
|
|
||||||
|
assert "insert into public." not in source
|
||||||
|
assert "update public." not in source
|
||||||
|
assert "delete from public." not in source
|
||||||
|
assert "systemctl restart" not in source
|
||||||
|
assert "send_message(" not in source
|
||||||
|
assert "guarded._approve" in source
|
||||||
|
assert "guarded._apply" in source
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_args_requires_one_explicit_command_boundary(tmp_path: Path) -> None:
|
||||||
|
args = lifecycle.parse_args(
|
||||||
|
[
|
||||||
|
"--container",
|
||||||
|
"teleo-disposable-clone",
|
||||||
|
"--db",
|
||||||
|
"teleo",
|
||||||
|
"--output",
|
||||||
|
str(tmp_path / "receipt.json"),
|
||||||
|
"--copy-model-auth",
|
||||||
|
"--operator-review",
|
||||||
|
"--guarded-apply",
|
||||||
|
]
|
||||||
|
)
|
||||||
|
|
||||||
|
assert args.container == "teleo-disposable-clone"
|
||||||
|
assert args.db == "teleo"
|
||||||
|
assert args.copy_model_auth is True
|
||||||
|
assert args.operator_review is True
|
||||||
|
assert args.guarded_apply is True
|
||||||
|
assert lifecycle.MARKER_RE.fullmatch(args.run_marker)
|
||||||
|
assert lifecycle.MARKER_RE.fullmatch(args.conversation_marker)
|
||||||
|
assert args.run_marker != args.conversation_marker
|
||||||
198
tests/test_stage_normalized_proposal.py
Normal file
198
tests/test_stage_normalized_proposal.py
Normal file
|
|
@ -0,0 +1,198 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import copy
|
||||||
|
import hashlib
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
REPO_ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
sys.path.insert(0, str(REPO_ROOT / "scripts"))
|
||||||
|
|
||||||
|
import stage_normalized_proposal as stage # noqa: E402
|
||||||
|
|
||||||
|
PARENT_ID = "77777777-7777-4777-8777-777777777701"
|
||||||
|
AGENT_ID = "11111111-1111-4111-8111-111111111111"
|
||||||
|
DOCUMENT = (
|
||||||
|
"Operating note: a knowledge change becomes canonical only after reviewer approval and guarded apply. "
|
||||||
|
"Review alone does not change canonical rows."
|
||||||
|
)
|
||||||
|
POST = "The moment a reviewer approves a Leo proposal, it is already canonical. No apply step is needed."
|
||||||
|
|
||||||
|
|
||||||
|
def _parent() -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"id": PARENT_ID,
|
||||||
|
"proposal_type": "attach_evidence",
|
||||||
|
"status": "pending_review",
|
||||||
|
"proposed_by_handle": "leo",
|
||||||
|
"proposed_by_agent_id": AGENT_ID,
|
||||||
|
"channel": "clone_composition",
|
||||||
|
"source_ref": "composition-fixture",
|
||||||
|
"payload": {
|
||||||
|
"claim_candidates": [
|
||||||
|
{
|
||||||
|
"claim_key": "guarded_apply_required",
|
||||||
|
"text": "Approval alone does not make a knowledge change canonical; guarded apply is required.",
|
||||||
|
"type": "structural",
|
||||||
|
"confidence": 0.98,
|
||||||
|
"evidence": [{"source_key": "operating_document", "role": "grounds"}],
|
||||||
|
"edges": [
|
||||||
|
{
|
||||||
|
"edge_type": "contradicts",
|
||||||
|
"target": "approval_is_canonical_post",
|
||||||
|
}
|
||||||
|
],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"claim_key": "approval_is_canonical_post",
|
||||||
|
"text": "A reviewer approval immediately makes a Leo proposal canonical.",
|
||||||
|
"type": "empirical",
|
||||||
|
"confidence": 0.4,
|
||||||
|
"evidence": [{"source_key": "contradictory_post", "role": "grounds"}],
|
||||||
|
},
|
||||||
|
],
|
||||||
|
"source_candidates": [
|
||||||
|
{
|
||||||
|
"source_key": "operating_document",
|
||||||
|
"source_type": "article",
|
||||||
|
"storage_path": "/composition/operating-note.txt",
|
||||||
|
"title": "Operating note",
|
||||||
|
"source_quality": "internal_operating_document",
|
||||||
|
"key_excerpt": DOCUMENT,
|
||||||
|
"content_sha256": hashlib.sha256(DOCUMENT.encode()).hexdigest(),
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"source_key": "contradictory_post",
|
||||||
|
"source_type": "post",
|
||||||
|
"url": "https://x.example.test/composition/1",
|
||||||
|
"title": "Contradictory post",
|
||||||
|
"source_quality": "unverified_social_post",
|
||||||
|
"key_excerpt": POST,
|
||||||
|
"content_sha256": hashlib.sha256(POST.encode()).hexdigest(),
|
||||||
|
},
|
||||||
|
],
|
||||||
|
"dedupe_conflict_assessment": {
|
||||||
|
"duplicate_claim_keys": [],
|
||||||
|
"conflicts": [
|
||||||
|
{
|
||||||
|
"from_claim_key": "guarded_apply_required",
|
||||||
|
"to_claim_key": "approval_is_canonical_post",
|
||||||
|
"relationship": "contradicts",
|
||||||
|
}
|
||||||
|
],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_prepare_normalized_child_is_deterministic_hash_bound_and_applyable() -> None:
|
||||||
|
first = stage.prepare_normalized_child(_parent())
|
||||||
|
second = stage.prepare_normalized_child(copy.deepcopy(_parent()))
|
||||||
|
|
||||||
|
assert first == second
|
||||||
|
assert first["status"] == "pending_review"
|
||||||
|
assert first["proposal_type"] == "approve_claim"
|
||||||
|
assert first["source_ref"].startswith(f"normalized:{PARENT_ID}:")
|
||||||
|
payload = first["payload"]["apply_payload"]
|
||||||
|
assert len(payload["claims"]) == 2
|
||||||
|
assert len(payload["sources"]) == 4
|
||||||
|
assert len(payload["evidence"]) == 4
|
||||||
|
assert len(payload["edges"]) == 1
|
||||||
|
assert payload["edges"][0]["edge_type"] == "contradicts"
|
||||||
|
source_hashes = {row["hash"] for row in payload["sources"]}
|
||||||
|
assert hashlib.sha256(DOCUMENT.encode()).hexdigest() in source_hashes
|
||||||
|
assert hashlib.sha256(POST.encode()).hexdigest() in source_hashes
|
||||||
|
assert payload["normalization_manifest"]["parent_packet_sha256"] == stage.canonical_sha256(_parent())
|
||||||
|
|
||||||
|
|
||||||
|
def test_invalid_source_hash_refuses_to_prepare_a_child() -> None:
|
||||||
|
parent = _parent()
|
||||||
|
parent["payload"]["source_candidates"][0]["content_sha256"] = "bad"
|
||||||
|
|
||||||
|
with pytest.raises(stage.bound.CheckpointError, match="did not normalize"):
|
||||||
|
stage.prepare_normalized_child(parent)
|
||||||
|
|
||||||
|
|
||||||
|
def test_stage_sql_validates_exact_pending_row_before_commit_and_never_writes_public() -> None:
|
||||||
|
child = stage.prepare_normalized_child(_parent())
|
||||||
|
sql = stage.build_stage_sql(child)
|
||||||
|
lowered = sql.lower()
|
||||||
|
|
||||||
|
assert "insert into kb_stage.kb_proposals" in lowered
|
||||||
|
assert "on conflict (id) do nothing" in lowered
|
||||||
|
assert "normalized stage validation failed" in lowered
|
||||||
|
assert lowered.index("raise exception") < lowered.index("commit;")
|
||||||
|
assert child["id"] in sql
|
||||||
|
assert child["payload_sha256"] in sql
|
||||||
|
assert "insert into public." not in lowered
|
||||||
|
assert "update public." not in lowered
|
||||||
|
assert "delete from public." not in lowered
|
||||||
|
|
||||||
|
|
||||||
|
def test_stage_normalized_child_binds_to_full_disposable_container_id(
|
||||||
|
monkeypatch: pytest.MonkeyPatch,
|
||||||
|
) -> None:
|
||||||
|
target = {
|
||||||
|
"id": "clone-container-id",
|
||||||
|
"name": "composition-clone",
|
||||||
|
"labels": {stage.bound.DISPOSABLE_LABEL_KEY: stage.bound.DISPOSABLE_LABEL_VALUE},
|
||||||
|
"network_mode": "none",
|
||||||
|
"ports": {},
|
||||||
|
"mount_sources": ["/clone-data"],
|
||||||
|
}
|
||||||
|
production = {
|
||||||
|
"id": "production-container-id",
|
||||||
|
"name": stage.bound.PRODUCTION_CONTAINER,
|
||||||
|
"labels": {},
|
||||||
|
"network_mode": "bridge",
|
||||||
|
"ports": {},
|
||||||
|
"mount_sources": ["/production-data"],
|
||||||
|
}
|
||||||
|
monkeypatch.setattr(
|
||||||
|
stage.bound,
|
||||||
|
"container_identity",
|
||||||
|
lambda name: target if name == "composition-clone" else production,
|
||||||
|
)
|
||||||
|
observed: dict[str, Any] = {}
|
||||||
|
|
||||||
|
def fake_psql(container: str, database: str, sql: str) -> dict[str, Any]:
|
||||||
|
child = stage.prepare_normalized_child(_parent())
|
||||||
|
observed.update({"container": container, "database": database, "sql": sql})
|
||||||
|
return {
|
||||||
|
"created": True,
|
||||||
|
"proposal_id": child["id"],
|
||||||
|
"proposal_type": child["proposal_type"],
|
||||||
|
"status": "pending_review",
|
||||||
|
"source_ref": child["source_ref"],
|
||||||
|
"payload_sha256": child["payload_sha256"],
|
||||||
|
}
|
||||||
|
|
||||||
|
monkeypatch.setattr(stage.bound, "_psql_json", fake_psql)
|
||||||
|
|
||||||
|
result = stage.stage_normalized_child("composition-clone", "teleo", _parent())
|
||||||
|
|
||||||
|
assert observed["container"] == "clone-container-id"
|
||||||
|
assert observed["database"] == "teleo"
|
||||||
|
assert result["bound_container_id"] == "clone-container-id"
|
||||||
|
assert result["database_receipt"]["created"] is True
|
||||||
|
|
||||||
|
|
||||||
|
def test_main_rejects_the_production_container_without_reading_the_packet(tmp_path: Path) -> None:
|
||||||
|
missing = tmp_path / "missing.json"
|
||||||
|
|
||||||
|
assert (
|
||||||
|
stage.main(
|
||||||
|
[
|
||||||
|
"--container",
|
||||||
|
stage.bound.PRODUCTION_CONTAINER,
|
||||||
|
"--db",
|
||||||
|
stage.bound.PRODUCTION_DB,
|
||||||
|
"--proposal-file",
|
||||||
|
str(missing),
|
||||||
|
]
|
||||||
|
)
|
||||||
|
== 2
|
||||||
|
)
|
||||||
Loading…
Reference in a new issue