Merge pull request #75 from living-ip/codex/working-leo-live-clone-20260710
Some checks are pending
CI / lint-and-test (push) Waiting to run

Prove guarded Leo source composition on VPS clone
This commit is contained in:
twentyOne2x 2026-07-11 08:46:10 +02:00 committed by GitHub
commit a4462b1fbe
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
18 changed files with 11412 additions and 70 deletions

View file

@ -109,6 +109,112 @@ live count/service endpoints without writing them, and independently prove the
container/workdir are absent afterward. Do not install the candidate code into
the live Leo deploy merely to run this canary.
To prove that Leo itself can inspect one lifecycle state through the real
`GatewayRunner` while remaining bound to a disposable full-data clone, run the
checkpoint on the VPS as `teleo` while that clone exists:
```bash
sudo -u teleo install -d -m 700 /home/teleo/leo-checkpoint-reports
sudo -u teleo HOME=/home/teleo \
/home/teleo/.hermes/hermes-agent/venv/bin/python \
scripts/run_leo_clone_bound_handler_checkpoint.py \
--container <disposable-container> \
--db teleo \
--prompt-id <stable-id> \
--prompt "<Cory-style KB question>" \
--expected-state approved \
--copy-model-auth \
--output /home/teleo/leo-checkpoint-reports/leo-clone-bound-checkpoint.json
```
The supplied clone must have Docker label
`com.livingip.leo.checkpoint=disposable`, network mode `none`, no published
ports, no mount source shared with production, and a PostgreSQL system
identifier distinct from production. The harness resolves the name once, pins
all operations to the full container ID, and fails if the name is rebound.
`--copy-model-auth` copies only `auth.json` into a private UUID-scoped profile
and binds only the configured provider's env-backed credential in memory. The
receipt records provider/variable names, never credential values or hashes. The
gateway runs in a dedicated process group with no delivery adapters and only
`skills_list`, `skill_view`, and a terminal handler restricted to the temporary
clone-bound wrapper's read-only verbs. The terminal subprocess must not inherit
the model credential. A timeout must terminate the child process group before
the profile is removed.
The checkpoint must observe at least one successful `teleo-kb` call bound to
the supplied container/database, remove the profile, and prove production row
counts plus row-content fingerprints, gateway state, and live bridge hashes are
unchanged. Every model tool call must be in the allowlist and every KB call must
complete successfully for a clean pass. A correct answer after a failed call
and retry is recovered behavior, not a clean reliability pass. The checkpoint
does not review, approve, apply, restart, post to Telegram, or mutate
production.
After that read-only checkpoint is clean, prove the lower-level
pending-to-canonical lifecycle in the same kind of disposable clone:
```bash
sudo -u teleo HOME=/home/teleo \
/home/teleo/.hermes/hermes-agent/venv/bin/python \
scripts/run_leo_clone_lifecycle_checkpoint.py \
--container <disposable-container> \
--db teleo \
--copy-model-auth \
--operator-review \
--guarded-apply \
--output /home/teleo/leo-checkpoint-reports/leo-clone-lifecycle-checkpoint.json
```
This command gives Leo only one deterministic staging verb. The harness, not
Leo, owns review and guarded apply. A pass requires exact structured receipts
for conversation memory, pending, approved-but-unapplied, and applied state;
exact linked claim/source/evidence rows; an isolated handler reopen using the
same persisted session; zero rejected or nonzero terminal calls; and unchanged
production DB fingerprints, service state, and live bridge hashes. It is a
mutation primitive, not proof that Leo can extract knowledge from arbitrary
documents or tweets.
For the real source-composition checkpoint, first create a fresh full-data
clone with the same disposable label/network/mount rules, then install the gate
and separated ephemeral clone credentials into a private run directory:
```bash
sudo -u teleo HOME=/home/teleo \
/home/teleo/.hermes/hermes-agent/venv/bin/python \
scripts/bootstrap_clone_kb_gate.py \
--container <fresh-disposable-container> \
--db teleo \
--output-dir <private-run-directory>
```
The bootstrap must verify both `kb_review` and `kb_apply` logins and emit only
credential file paths, never secret values or hashes. Then run:
```bash
sudo -u teleo HOME=/home/teleo \
/home/teleo/.hermes/hermes-agent/venv/bin/python \
scripts/run_leo_clone_composition_checkpoint.py \
--container <fresh-disposable-container> \
--db teleo \
--copy-model-auth \
--operator-review \
--guarded-apply \
--review-secrets-file <private-run-directory>/kb-review.env \
--apply-secrets-file <private-run-directory>/kb-apply.env \
--output <private-run-directory>/reports/composition-current.json
```
A pass requires model-driven dedupe search and extraction from supplied source
bytes, exact source hashes and excerpts, linked claims/evidence/conflict edges,
no row change before staging, deterministic normalization, separated review and
apply, exact clone deltas, a new handler process reasoning over exact canonical
rows without supplied IDs, session-marker recall, nonce-bound tool receipts,
unchanged production DB/service/bridge state, and complete clone/profile/secret
cleanup. The retained current receipt is
`docs/reports/leo-working-state-20260709/leo-source-composition-clone-checkpoint-current.json`.
It is not Telegram-visible proof and does not authorize production apply.
## Blocker Format
Do not say "blocked" without:

View file

@ -27,12 +27,25 @@ A working Leo is a Telegram-facing agent that:
3. Distinguishes `proposed`, `pending_review`, `approved`, `applied`, and `not applied`.
4. Does not say an approval changed canonical DB state when only `kb_stage` changed.
5. Can answer vague, high-level Cory-style incident prompts without being spoon-fed exact proposal IDs.
6. Can stage concrete KB changes from Telegram with enough structure for review.
7. Can move approved concrete changes through a guarded apply path when authorized.
8. Retains Cory's caveats and review notes in source/evidence/proposal rows.
9. Produces before/after table-level proof and service stability readback.
10. Survives an intentional `leoclean-gateway.service` restart: active before, active after, canonical KB counts unchanged, and a no-post handler smoke still answers.
11. For a reviewed graph bundle, produces exact payload-controlled row
6. Can compose the KB from a previously unindexed document, URL, or tweet-like
source: retain a byte/hash-bound source locator, extract atomic claims and
exact evidence excerpts, preserve useful metadata, and link every claim to
its evidence and source.
7. Detects duplicates, conflicts, updates, and insufficient evidence before
staging; uncertainty must remain visible instead of being normalized into a
stronger claim.
8. Can stage those concrete KB changes from Telegram with enough structure for
human review, without making staged content canonical.
9. Can move approved concrete changes through a guarded apply path when authorized.
10. Retains Cory's caveats and review notes in source/evidence/proposal rows.
11. Reasons over claims, evidence, sources, edges, and open conflicts as a graph,
and can explain which rows support or weaken an answer without being given IDs.
12. Rebuilds any compiled identity/workspace artifact deterministically from
canonical DB rows, reports the source rows and freshness boundary, and does
not treat edits to a generated artifact as canonical knowledge.
13. Produces before/after table-level proof and service stability readback.
14. Survives an intentional `leoclean-gateway.service` restart: active before, active after, canonical KB counts unchanged, and a no-post handler smoke still answers.
15. For a reviewed graph bundle, produces exact payload-controlled row
projections, exact table deltas, source-byte binding, and cleanup proof in a
disposable runtime before any production permission or apply window.
@ -46,6 +59,26 @@ A working Leo is a Telegram-facing agent that:
- Old rich proposal packets: `14fa5ecc...`, `ac036c9d...`, and `a64df080...` are not to be silently production-applied.
- Guarded apply proof: both generic and real Helmer v3 receipts pass `37/37` in
disposable PostgreSQL; production Helmer remains unapplied.
- Clone-bound handler proof: the real VPS `GatewayRunner` can inspect the full
current-data disposable clone, discover the Helmer proposal, distinguish
`approved` from `applied`, and produce bound bridge-call evidence without a
Telegram post or production change. The proof surface must expose no delivery
adapters or send tool, restrict terminal execution to clone-bound read-only KB
verbs, kill the handler process group on timeout, and compare production
row-content fingerprints before and after. Treat open-ended latency and
retries as a separate reliability dimension; a correct answer after a failed
tool call is recovered behavior, not a clean pass, and one correct answer does
not establish a stable pass rate.
- Source-composition proof: the real VPS `GatewayRunner` passed `34/34` in a
fresh no-send full-data clone. It searched existing knowledge, extracted two
hash-bound conflicting claims from a new document and post, staged a strict
proposal, preserved an immutable separated approval, applied exact canonical
rows, reopened in a new child process, recalled the prior marker, discovered
the new proposal/claims without supplied IDs, read exact evidence/source UUIDs
and edges, and explained `approved != applied`. Production DB fingerprints,
service PID/restarts, and live bridge hashes stayed unchanged; all disposable
containers, volumes, profiles, credentials, and run directories were removed.
This is clone proof, not Telegram delivery or production apply proof.
## Cory Direct Questions
@ -90,7 +123,15 @@ Leo is not working just because:
Those are evidence. The target is Telegram-visible behavior plus DB-state truth plus a guarded path to canonical rows.
A synthetic one-claim proposal proves only the guarded mutation primitive. It
does not prove KB composition. Composition requires a source that was absent at
baseline, model-driven extraction into linked source/evidence/claim proposals,
dedupe/conflict readback, guarded canonical apply, and an open-ended answer
grounded in the new rows after an isolated handler restart.
## Proof Files
Use `docs/reports/leo-working-state-20260709/current-truth-index.md` for current artifact paths.
Use `docs/reports/leo-working-state-20260709/leo-source-composition-clone-checkpoint-current.md`
and its JSON companion for the current source-composition claim ceiling.
For restart survival, use `scripts/collect_leo_restart_survival_proof.py --execute-restart`; expected artifacts are `docs/reports/leo-working-state-20260709/leo-restart-survival-proof-current.json` and `.md`.

View file

@ -61,6 +61,8 @@ Use this file before making status claims. Prefer fresh VPS/GCP readbacks when c
- Lossless Helmer strict normalization: `docs/reports/leo-working-state-20260709/helmer-approve-claim-normalization-current.json`
- Approved claim-bundle lifecycle harness: `scripts/run_approve_claim_clone_canary.py`
- Approved claim-bundle isolated-container wrapper: `scripts/run_approve_claim_isolated_container_canary.sh`
- Live VPS source-composition clone checkpoint summary: `docs/reports/leo-working-state-20260709/leo-source-composition-clone-checkpoint-current.md`
- Live VPS source-composition clone checkpoint full JSON: `docs/reports/leo-working-state-20260709/leo-source-composition-clone-checkpoint-current.json`
- Claim/source preview: `docs/reports/leo-working-state-20260709/claim-source-contract-preview-current.md`
- Mapped rich proposal plan: `docs/reports/leo-working-state-20260709/rich-proposal-creation-plan-mapped-current.md`
- Mapped production apply packet, not executed in production: `docs/reports/leo-working-state-20260709/production-apply-packet-current.md`
@ -161,6 +163,7 @@ Use this file before making status claims. Prefer fresh VPS/GCP readbacks when c
- The highest-risk direct-claim cases are now green at handler level: `DC-02` recognizes approved Helmer staging row `a64df080-8502-42e2-98f4-9bbdecb8da73` as approved/unapplied, `DC-03` says the decision-matrix path is not shipped while matrix tables are absent, `DC-04` avoids a single-cause document pointer diagnosis, `DC-05` separates staging demo from authorized canonical apply, and `DC-06` separates `SOUL.md` runtime edits from canonical DB identity.
- A strict canonical `add_edge` apply canary is live-proven.
- The repo-owned normal apply path now supports an atomic, insert-only `approve_claim` bundle behind separate `kb_review` and `kb_apply` credentials, a DB-role-to-reviewer principal mapping, and an immutable approval snapshot. Final source-bound isolated-container receipts pass `37/37` for both the generic bundle (`2` claims, `2` sources, `2` evidence rows, `1` edge, `1` reasoning tool) and the real losslessly normalized Helmer bundle (`5/13/17/7/1`). They prove exact payload-controlled projections and table deltas, reviewer/apply separation, denied direct proposal/evidence mutation, stale-payload refusal before canonical writes, replay refusal, source-hash rollback, exact agent/external-claim seeding, hardened role/function/ACL state, and independent inner/outer cleanup. PR #72 source later auto-synchronized to the VPS checkout and deploy stamp at merge commit `771679061f0a0b6463fb7d9cf530eb9d2d4e5440`; the timer journal says no Python changes and no service restart, the commit had no delta under the Leo profile source paths, the gateway stayed PID `2999690`/`NRestarts=0`, the apply worker stayed disabled/inactive, and read-only count endpoints remained `1837/4145/4670/4916/17/26`. This is a source checkout sync, not proof that the production permission migration ran, the worker was enabled, canonical rows changed, or byte-for-byte live content stayed equal. Helmer remains unapplied on production. See `pr72-vps-auto-deploy-runtime-nonchange-current.md`.
- The live VPS source-composition checkpoint now passes `34/34` with zero errors on a fresh no-send full-data clone: Leo searched the current KB, extracted two atomic conflicting claims from a new hash-bound document and post, retained exact evidence/source links, staged a strict proposal, passed separated review and guarded apply, then reopened in a new handler process and discovered/reasoned over the exact canonical proposal, claims, source UUIDs, and conflict edge without supplied IDs while recalling the prior conversation marker. Production counts and row-content fingerprints, gate schema, gateway PID/restarts, and live bridge hashes stayed unchanged. All three experiment containers, anonymous volumes, private credentials, run directories, temporary profiles, and child processes were removed. This proves current-data clone composition and restarted reasoning, not Telegram-visible delivery, production apply, scheduled DB-to-identity recomposition, arbitrary-source breadth, or GCP parity. See `leo-source-composition-clone-checkpoint-current.md`.
- Helmer 7 Powers is full-coverage packet and ledger clone-proven, not production-applied.
- Mapped rich approved proposal packet is clone/rehearsal-proven for the supported subset, not production-applied.
- The strategy-anchor companion write for mapped rich proposal `14fa5ecc` is clone-proven and reversible, not production-applied.

View file

@ -0,0 +1,68 @@
# Leo Source Composition Clone Checkpoint
## Result
- Status: `pass`, `34/34` checks, zero harness errors.
- Tier: live VPS, no-send, disposable full-data clone.
- Full receipt: `leo-source-composition-clone-checkpoint-current.json`.
- Receipt SHA-256: `897f2b48a5b85a32a069e154f246140421a16942acd85e32e4b8edc1d6738278`.
- Production behavior changed: no.
## What Ran
Leo received a previously unindexed operating document and a contradictory
post for `AURORA-A1CF3DF21D`. The model first searched the current clone KB,
then extracted two atomic claims, retained both supplied source-content hashes,
linked evidence from both sources, and represented the conflict bidirectionally.
Extraction changed no database rows.
The harness normalized the packet into strict proposal
`b84244ce-53c8-5172-8c5e-38814f4f4e95`, staged it as `pending_review`, performed
a separated operator review to `approved`, and ran guarded apply to `applied`.
All mutation occurred only in disposable clone `leo-compose-20260711-03`.
Exact clone deltas were:
- `kb_stage.kb_proposals`: `+1`
- `kb_stage.kb_proposal_approvals`: `+1`
- `public.claims`: `+2`
- `public.sources`: `+4`
- `public.claim_evidence`: `+4`
- `public.claim_edges`: `+2`
- `public.reasoning_tools`: `+0`
## Restarted Reasoning
A new isolated GatewayRunner child reused the same private persisted session,
recalled `composition-memory-live-20260711-03`, searched without generated IDs,
found and opened the applied proposal, read JSON evidence for both claims, read
the canonical contradiction edge, and returned every exact proposal, claim,
source, and edge ID in its final state receipt.
The transcript contained seven terminal calls. Every call had a result, none
was rejected or nonzero, all seven matched nonce-bound wrapper receipts, and the
two bounded/truncated transcript results reconciled to successful wrapper
receipts. The GatewayRunner exposed no delivery adapter or send tool.
## Production And Cleanup
Production remained byte-for-byte stable across the guarded fingerprints:
- gateway: PID `2999690`, `NRestarts=0`, start time unchanged;
- canonical counts: claims `1837`, sources `4145`, evidence `4670`, edges
`4916`, reasoning tools `17`, proposals `26`;
- every tracked production table MD5 unchanged;
- review/apply gate schema unchanged and still absent from production;
- live KB wrapper and bridge-skill hashes unchanged.
All three composition containers, their anonymous volumes, private credential
files, run directories, temporary profiles, and GatewayRunner children were
removed and independently verified absent.
## Claim Ceiling
This proves source extraction, hash-bound normalization, guarded canonical
composition, conflict-aware reasoning, and isolated handler/session survival on
a current full-data VPS clone. It does not prove Telegram-visible delivery,
production DB mutation, scheduled DB-to-identity recomposition, arbitrary-source
coverage beyond this case, or GCP parity.

View file

@ -0,0 +1,209 @@
#!/usr/bin/env python3
"""Install guarded review/apply prerequisites and ephemeral credentials in a disposable clone."""
from __future__ import annotations
import argparse
import json
import os
import secrets
import subprocess
import sys
from pathlib import Path
from typing import Any
HERE = Path(__file__).resolve().parent
sys.path.insert(0, str(HERE))
import apply_proposal as ap # noqa: E402
import run_leo_clone_bound_handler_checkpoint as bound # noqa: E402
def _psql(
container: str,
database: str,
sql: str,
*,
secret_values: tuple[str, ...] = (),
) -> None:
command = [
bound.SYSTEM_DOCKER,
"exec",
"-i",
container,
"psql",
"-U",
"postgres",
"-d",
database,
"-v",
"ON_ERROR_STOP=1",
"-At",
"-q",
]
result = subprocess.run(
command,
input=sql,
text=True,
capture_output=True,
env={"PATH": bound.SYSTEM_EXEC_PATH},
check=False,
)
if result.returncode != 0:
raise bound.CheckpointError(
f"clone gate SQL failed ({result.returncode}): "
+ bound.redact_text(result.stderr or result.stdout, secret_values=secret_values)
)
def _verify_login(container: str, database: str, role: str, password: str) -> None:
command = [
bound.SYSTEM_DOCKER,
"exec",
"-e",
"PGPASSWORD",
"-i",
container,
"psql",
"-U",
role,
"-h",
"127.0.0.1",
"-d",
database,
"-v",
"ON_ERROR_STOP=1",
"-At",
"-q",
"-c",
"select current_user;",
]
result = subprocess.run(
command,
text=True,
capture_output=True,
env={"PATH": bound.SYSTEM_EXEC_PATH, "PGPASSWORD": password},
check=False,
)
if result.returncode != 0 or result.stdout.strip() != role:
raise bound.CheckpointError(f"ephemeral {role} clone login verification failed")
def _write_secret(path: Path, key: str, value: str) -> None:
descriptor = os.open(
path,
os.O_WRONLY | os.O_CREAT | os.O_EXCL | getattr(os, "O_NOFOLLOW", 0),
0o600,
)
with os.fdopen(descriptor, "w", encoding="utf-8") as handle:
handle.write(f"{key}={value}\n")
handle.flush()
os.fsync(handle.fileno())
def bootstrap(container: str, database: str, output_dir: Path) -> dict[str, Any]:
target = bound.container_identity(container)
production = bound.container_identity(bound.PRODUCTION_CONTAINER)
isolation = bound.validate_disposable_target_identity(target, production)
target_id = str(target["id"])
output_dir = output_dir.resolve(strict=True)
stat = output_dir.stat()
if not output_dir.is_dir() or stat.st_uid != os.geteuid() or stat.st_mode & 0o077:
raise bound.CheckpointError("--output-dir must be an existing private directory owned by the current user")
review_file = output_dir / "kb-review.env"
apply_file = output_dir / "kb-apply.env"
if review_file.exists() or review_file.is_symlink() or apply_file.exists() or apply_file.is_symlink():
raise bound.CheckpointError("clone credential files already exist; use a fresh private run directory")
review_password = secrets.token_urlsafe(36)
apply_password = secrets.token_urlsafe(36)
prereqs = (HERE / "kb_apply_prereqs.sql").read_text(encoding="utf-8")
_psql(
target_id,
database,
"""do $bootstrap$
begin
if not exists (select 1 from pg_roles where rolname = 'kb_apply') then
create role kb_apply login;
end if;
end
$bootstrap$;
""",
)
_psql(target_id, database, prereqs)
_psql(
target_id,
database,
"alter role kb_review password "
+ ap.sql_literal(review_password)
+ ";\nalter role kb_apply password "
+ ap.sql_literal(apply_password)
+ ";\n",
secret_values=(review_password, apply_password),
)
_verify_login(target_id, database, "kb_review", review_password)
_verify_login(target_id, database, "kb_apply", apply_password)
manifest = bound._psql_json(
target_id,
database,
"""\\set ON_ERROR_STOP on
begin transaction read only;
select jsonb_build_object(
'approval_table', to_regclass('kb_stage.kb_proposal_approvals')::text,
'review_principals_table', to_regclass('kb_stage.kb_review_principals')::text,
'review_role_exists', exists(select 1 from pg_roles where rolname = 'kb_review'),
'apply_role_exists', exists(select 1 from pg_roles where rolname = 'kb_apply')
)::text;
rollback;
""",
)
if manifest != {
"approval_table": "kb_stage.kb_proposal_approvals",
"review_principals_table": "kb_stage.kb_review_principals",
"review_role_exists": True,
"apply_role_exists": True,
}:
raise bound.CheckpointError("clone gate manifest is incomplete: " + json.dumps(manifest, sort_keys=True))
_write_secret(review_file, "KB_REVIEW_PASSWORD", review_password)
try:
_write_secret(apply_file, "KB_APPLY_DB_PASSWORD", apply_password)
except BaseException:
review_file.unlink(missing_ok=True)
raise
review_password = ""
apply_password = ""
return {
"status": "ready",
"bound_container_id": target_id,
"database": database,
"isolation_checks": isolation,
"gate_manifest": manifest,
"review_secret_file": str(review_file),
"apply_secret_file": str(apply_file),
"secret_values_or_hashes_recorded": False,
"login_roles_verified": ["kb_review", "kb_apply"],
}
def parse_args(argv: list[str] | None = None) -> argparse.Namespace:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--container", required=True)
parser.add_argument("--db", required=True)
parser.add_argument("--output-dir", required=True, type=Path)
return parser.parse_args(argv)
def main(argv: list[str] | None = None) -> int:
args = parse_args(argv)
try:
if args.container == bound.PRODUCTION_CONTAINER:
raise bound.CheckpointError("--container must be a disposable non-production target")
result = bootstrap(args.container, args.db, args.output_dir)
except (OSError, bound.CheckpointError) as exc:
print(json.dumps({"status": "rejected", "error": str(exc)}, sort_keys=True))
return 2
print(json.dumps(result, sort_keys=True))
return 0
if __name__ == "__main__":
raise SystemExit(main())

View file

@ -33,9 +33,8 @@ import apply_proposal as ap # noqa: E402
import kb_proposal_review_packet as review_packet # noqa: E402
import kb_rich_proposal_creation_plan as rich_plan # noqa: E402
UUID_RE = re.compile(
r"^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$"
)
UUID_RE = re.compile(r"^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$")
SHA256_RE = re.compile(r"^[0-9a-fA-F]{64}$")
RICH_CANDIDATE_COLLECTIONS = (
("claim_candidates", "claim_key", "claim"),
("proposed_claims", "claim_key", "claim"),
@ -154,6 +153,21 @@ def _candidate_inventory(payload: dict[str, Any]) -> dict[str, Any]:
inventory["blocked_fragments"].append(block)
continue
if kind == "source":
supplied_hash = candidate.get("content_sha256") or candidate.get("hash")
if supplied_hash is not None and (
not isinstance(supplied_hash, str) or not SHA256_RE.fullmatch(supplied_hash)
):
block = _blocked(
"source_candidate",
f"{location} has an invalid content_sha256/hash; expected 64 hexadecimal characters",
["content_sha256"],
candidate,
)
block["location"] = location
inventory["blocked_fragments"].append(block)
continue
inventory[f"{kind}_items"].append(
{
"collection": collection,
@ -183,9 +197,7 @@ def _candidate_inventory(payload: dict[str, Any]) -> dict[str, Any]:
return inventory
def _planned_row_accounting(
plan: dict[str, Any], inventory: dict[str, Any]
) -> tuple[dict[str, Any], list[str]]:
def _planned_row_accounting(plan: dict[str, Any], inventory: dict[str, Any]) -> tuple[dict[str, Any], list[str]]:
"""Reconcile every validated input candidate to exactly one planned row."""
rows = plan.get("planned_rows") if isinstance(plan.get("planned_rows"), dict) else {}
generated = plan.get("generated_ids") if isinstance(plan.get("generated_ids"), dict) else {}
@ -264,9 +276,7 @@ def _normalization_manifest(plan: dict[str, Any], accounting: dict[str, Any]) ->
for row in rows["claims"]:
mapping = row.get("type_normalization")
if mapping:
claim_type_mappings.append(
{"claim_key": row.get("claim_key"), "claim_id": row.get("id"), **dict(mapping)}
)
claim_type_mappings.append({"claim_key": row.get("claim_key"), "claim_id": row.get("id"), **dict(mapping)})
for row in rows["sources"]:
mapping = row.get("source_type_normalization")
if mapping:
@ -345,17 +355,11 @@ def _strict_approve_claim_child(
"source_proposal_id": proposal.get("id"),
"agent_id": agent_id,
"claims": [
{
key: row.get(key)
for key in ("id", "type", "text", "status", "confidence", "tags", "created_by")
}
{key: row.get(key) for key in ("id", "type", "text", "status", "confidence", "tags", "created_by")}
for row in rows["claims"]
],
"sources": [
{
key: row.get(key)
for key in ("id", "source_type", "url", "storage_path", "excerpt", "hash", "created_by")
}
{key: row.get(key) for key in ("id", "source_type", "url", "storage_path", "excerpt", "hash", "created_by")}
for row in rows["sources"]
],
"evidence": [
@ -379,10 +383,7 @@ def _strict_approve_claim_child(
for row in rows["claim_edges"]
],
"reasoning_tools": [
{
key: row.get(key)
for key in ("id", "agent_id", "name", "description", "category")
}
{key: row.get(key) for key in ("id", "agent_id", "name", "description", "category")}
for row in rows["reasoning_tools"]
],
"normalization_manifest": normalization_manifest,
@ -497,13 +498,17 @@ def _normalize_evidence(payload: dict[str, Any]) -> tuple[list[dict[str, Any]],
missing.append("canonical claim_id")
if not _first_uuid(fragment, ("source_id", "source_uuid", "canonical_source_id")):
missing.append("canonical source_id")
blocked.append(_blocked("evidence", "evidence fragment is not a strict attach_evidence contract", missing, fragment))
blocked.append(
_blocked("evidence", "evidence fragment is not a strict attach_evidence contract", missing, fragment)
)
strict = [_strict_child("attach_evidence", {"evidence": evidence_rows}, evidence_rows)] if evidence_rows else []
return strict, blocked
def _normalize_strategy(proposal: dict[str, Any], payload: dict[str, Any]) -> tuple[list[dict[str, Any]], list[dict[str, Any]]]:
def _normalize_strategy(
proposal: dict[str, Any], payload: dict[str, Any]
) -> tuple[list[dict[str, Any]], list[dict[str, Any]]]:
if proposal.get("proposal_type") != "revise_strategy":
return [], []
strategy_payload = payload.get("apply_payload") or {
@ -534,9 +539,7 @@ def _dedupe_children(children: list[dict[str, Any]]) -> list[dict[str, Any]]:
return out
def normalize_proposal(
proposal: dict[str, Any], mappings: dict[str, Any] | None = None
) -> dict[str, Any]:
def normalize_proposal(proposal: dict[str, Any], mappings: dict[str, Any] | None = None) -> dict[str, Any]:
payload = proposal.get("payload") or {}
if not isinstance(payload, dict):
payload = {}

View file

@ -134,7 +134,14 @@ def _source_excerpt(source: dict[str, Any]) -> str:
metadata = {
key: source.get(key)
for key in ("title", "source_key", "source_quality", "usage_limits", "claim_id")
for key in (
"title",
"source_key",
"source_quality",
"usage_limits",
"claim_id",
"content_sha256",
)
if source.get(key) is not None
}
if metadata:
@ -157,7 +164,11 @@ def _normalize_source_type(raw_type: Any) -> tuple[str, dict[str, Any] | None]:
if raw_text in LIVE_SOURCE_TYPES:
return raw_text, None
mapped = SOURCE_TYPE_ALIASES.get(raw_text, "other")
return mapped, {"from": raw_text, "to": mapped, "reason": "proposal source_type collapsed to live sources.source_type"}
return mapped, {
"from": raw_text,
"to": mapped,
"reason": "proposal source_type collapsed to live sources.source_type",
}
def _claim_text(candidate: dict[str, Any]) -> str:
@ -177,8 +188,16 @@ def _normalize_claim_type(raw_type: Any) -> tuple[str, dict[str, Any] | None]:
return part, {"from": raw_text, "to": part, "reason": "proposal subtype collapsed to live claims.type"}
if part in CLAIM_TYPE_ALIASES:
mapped = CLAIM_TYPE_ALIASES[part]
return mapped, {"from": raw_text, "to": mapped, "reason": "proposal governance/process subtype mapped to meta"}
return "meta", {"from": raw_text, "to": "meta", "reason": "unsupported proposal type requires review; meta used for rehearsal"}
return mapped, {
"from": raw_text,
"to": mapped,
"reason": "proposal governance/process subtype mapped to meta",
}
return "meta", {
"from": raw_text,
"to": "meta",
"reason": "unsupported proposal type requires review; meta used for rehearsal",
}
def _claim_body_excerpt(proposal: dict[str, Any], candidate: dict[str, Any]) -> str:
@ -200,7 +219,9 @@ def _claim_body_excerpt(proposal: dict[str, Any], candidate: dict[str, Any]) ->
return _compact_text(json.dumps(body_packet, ensure_ascii=False, sort_keys=True))
def _candidate_claim_rows(proposal: dict[str, Any], payload: dict[str, Any]) -> tuple[list[dict[str, Any]], dict[str, str]]:
def _candidate_claim_rows(
proposal: dict[str, Any], payload: dict[str, Any]
) -> tuple[list[dict[str, Any]], dict[str, str]]:
rows: list[dict[str, Any]] = []
key_to_id: dict[str, str] = {}
proposal_id = str(proposal.get("id") or "")
@ -250,6 +271,7 @@ def _source_rows(
excerpt = _source_excerpt(source)
raw_source_type = _source_type(source)
source_type, source_type_normalization = _normalize_source_type(raw_source_type)
supplied_hash = source.get("content_sha256") or source.get("hash")
row = {
"id": source_id,
"source_key": source_key,
@ -259,12 +281,16 @@ def _source_rows(
"url": source.get("url"),
"storage_path": source.get("local_text") or source.get("local_snapshot") or source.get("storage_path"),
"excerpt": excerpt,
"hash": _stable_hash({"proposal_id": proposal_id, "source_key": source_key, "excerpt": excerpt}),
"hash": str(supplied_hash).lower()
if supplied_hash
else _stable_hash({"proposal_id": proposal_id, "source_key": source_key, "excerpt": excerpt}),
"created_by": created_by,
}
rows.append(row)
claim_candidates = {str(candidate.get("claim_key") or ""): candidate for candidate in _claim_candidate_items(payload)}
claim_candidates = {
str(candidate.get("claim_key") or ""): candidate for candidate in _claim_candidate_items(payload)
}
for claim in claim_rows:
claim_key = claim["claim_key"]
candidate = claim_candidates.get(claim_key)
@ -440,7 +466,15 @@ def _plan_evidence(
key = (claim_id, source_id, role)
if key not in seen:
seen.add(key)
planned.append({"claim_id": claim_id, "source_id": source_id, "role": role, "weight": None, "source_fragment": fragment})
planned.append(
{
"claim_id": claim_id,
"source_id": source_id,
"role": role,
"weight": None,
"source_fragment": fragment,
}
)
for candidate in _claim_candidate_items(payload):
claim_key = str(candidate.get("claim_key") or "")
@ -563,47 +597,47 @@ def _reasoning_tool_rows(proposal: dict[str, Any], payload: dict[str, Any]) -> l
def _reasoning_tool_insert_sql(row: dict[str, Any]) -> str:
return f"""insert into public.reasoning_tools (id, agent_id, name, description, category)
select {_sql_literal(row['id'])}::uuid, {_sql_literal(row['agent_id'])}::uuid,
{_sql_literal(row['name'])}, {_sql_literal(row['description'])}, {_sql_literal(row['category'])}
where not exists (select 1 from public.reasoning_tools where id = {_sql_literal(row['id'])}::uuid);"""
select {_sql_literal(row["id"])}::uuid, {_sql_literal(row["agent_id"])}::uuid,
{_sql_literal(row["name"])}, {_sql_literal(row["description"])}, {_sql_literal(row["category"])}
where not exists (select 1 from public.reasoning_tools where id = {_sql_literal(row["id"])}::uuid);"""
def _claim_insert_sql(row: dict[str, Any]) -> str:
return f"""insert into public.claims (id, type, text, status, confidence, tags, created_by)
select {_sql_literal(row['id'])}::uuid, {_sql_literal(row['type'])}, {_sql_literal(row['text'])},
{_sql_literal(row['status'])}, {_sql_literal(row['confidence'])}, {_sql_text_array(row['tags'])},
{_sql_literal(row['created_by'])}::uuid
where not exists (select 1 from public.claims where id = {_sql_literal(row['id'])}::uuid);"""
select {_sql_literal(row["id"])}::uuid, {_sql_literal(row["type"])}, {_sql_literal(row["text"])},
{_sql_literal(row["status"])}, {_sql_literal(row["confidence"])}, {_sql_text_array(row["tags"])},
{_sql_literal(row["created_by"])}::uuid
where not exists (select 1 from public.claims where id = {_sql_literal(row["id"])}::uuid);"""
def _source_insert_sql(row: dict[str, Any]) -> str:
return f"""insert into public.sources (id, source_type, url, storage_path, excerpt, hash, created_by)
select {_sql_literal(row['id'])}::uuid, {_sql_literal(row['source_type'])}, {_sql_literal(row['url'])},
{_sql_literal(row['storage_path'])}, {_sql_literal(row['excerpt'])}, {_sql_literal(row['hash'])},
{_sql_literal(row['created_by'])}::uuid
where not exists (select 1 from public.sources where id = {_sql_literal(row['id'])}::uuid);"""
select {_sql_literal(row["id"])}::uuid, {_sql_literal(row["source_type"])}, {_sql_literal(row["url"])},
{_sql_literal(row["storage_path"])}, {_sql_literal(row["excerpt"])}, {_sql_literal(row["hash"])},
{_sql_literal(row["created_by"])}::uuid
where not exists (select 1 from public.sources where id = {_sql_literal(row["id"])}::uuid);"""
def _edge_insert_sql(row: dict[str, Any]) -> str:
return f"""insert into public.claim_edges (from_claim, to_claim, edge_type, weight)
select {_sql_literal(row['from_claim'])}::uuid, {_sql_literal(row['to_claim'])}::uuid,
{_sql_literal(row['edge_type'])}::edge_type, {_sql_literal(row['weight'])}
select {_sql_literal(row["from_claim"])}::uuid, {_sql_literal(row["to_claim"])}::uuid,
{_sql_literal(row["edge_type"])}::edge_type, {_sql_literal(row["weight"])}
where not exists (
select 1 from public.claim_edges
where from_claim = {_sql_literal(row['from_claim'])}::uuid
and to_claim = {_sql_literal(row['to_claim'])}::uuid
and edge_type = {_sql_literal(row['edge_type'])}::edge_type);"""
where from_claim = {_sql_literal(row["from_claim"])}::uuid
and to_claim = {_sql_literal(row["to_claim"])}::uuid
and edge_type = {_sql_literal(row["edge_type"])}::edge_type);"""
def _evidence_insert_sql(row: dict[str, Any]) -> str:
return f"""insert into public.claim_evidence (claim_id, source_id, role, weight)
select {_sql_literal(row['claim_id'])}::uuid, {_sql_literal(row['source_id'])}::uuid,
{_sql_literal(row['role'])}::evidence_role, {_sql_literal(row['weight'])}
select {_sql_literal(row["claim_id"])}::uuid, {_sql_literal(row["source_id"])}::uuid,
{_sql_literal(row["role"])}::evidence_role, {_sql_literal(row["weight"])}
where not exists (
select 1 from public.claim_evidence
where claim_id = {_sql_literal(row['claim_id'])}::uuid
and source_id = {_sql_literal(row['source_id'])}::uuid
and role = {_sql_literal(row['role'])}::evidence_role);"""
where claim_id = {_sql_literal(row["claim_id"])}::uuid
and source_id = {_sql_literal(row["source_id"])}::uuid
and role = {_sql_literal(row["role"])}::evidence_role);"""
def build_rollback_sql(plan: dict[str, Any]) -> str:
@ -627,7 +661,9 @@ def build_creation_plan(proposal: dict[str, Any], mappings: dict[str, Any] | Non
claim_rows, claim_key_to_id = _candidate_claim_rows(proposal, payload)
source_rows, source_key_to_id, body_source_by_claim_key = _source_rows(proposal, payload, claim_rows)
edge_rows, blocked_edges = _plan_edges(payload, claim_key_to_id, mappings)
evidence_rows, blocked_evidence = _plan_evidence(payload, claim_key_to_id, source_key_to_id, body_source_by_claim_key)
evidence_rows, blocked_evidence = _plan_evidence(
payload, claim_key_to_id, source_key_to_id, body_source_by_claim_key
)
reasoning_tool_rows = _reasoning_tool_rows(proposal, payload)
old_claim = _payload_dict(payload, "old_claim")

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,221 @@
#!/usr/bin/env python3
"""Normalize one rich KB packet and stage its strict child in a disposable clone."""
from __future__ import annotations
import argparse
import hashlib
import json
import sys
import uuid
from pathlib import Path
from typing import Any
HERE = Path(__file__).resolve().parent
sys.path.insert(0, str(HERE))
import apply_proposal as ap # noqa: E402
import kb_proposal_normalize as normalize # noqa: E402
import run_leo_clone_bound_handler_checkpoint as bound # noqa: E402
def canonical_sha256(value: Any) -> str:
encoded = json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=True).encode()
return hashlib.sha256(encoded).hexdigest()
def _uuid(value: Any, label: str) -> str:
try:
return str(uuid.UUID(str(value)))
except ValueError as exc:
raise bound.CheckpointError(f"{label} must be a full UUID") from exc
def prepare_normalized_child(parent: dict[str, Any]) -> dict[str, Any]:
parent_id = _uuid(parent.get("id"), "parent proposal id")
preview = normalize.normalize_proposal(parent)
if preview.get("normalization_state") != "strict_children_ready" or preview.get("strict_child_count") != 1:
raise bound.CheckpointError(
"rich proposal did not normalize to exactly one strict child: "
+ json.dumps(
{
"normalization_state": preview.get("normalization_state"),
"strict_child_count": preview.get("strict_child_count"),
"blocked_fragments": preview.get("blocked_fragments"),
},
sort_keys=True,
)
)
strict = preview["strict_child_proposals"][0]
proposal_type = strict.get("proposal_type")
if proposal_type not in ap.APPLYABLE_TYPES:
raise bound.CheckpointError(f"normalized child type is not applyable: {proposal_type!r}")
payload = json.loads(json.dumps(strict.get("payload"), sort_keys=True))
apply_payload = payload.get("apply_payload") if isinstance(payload, dict) else None
if not isinstance(apply_payload, dict):
raise bound.CheckpointError("normalized child has no strict payload.apply_payload object")
manifest = apply_payload.setdefault("normalization_manifest", {})
if not isinstance(manifest, dict):
raise bound.CheckpointError("normalized child normalization_manifest must be an object")
parent_packet_sha256 = canonical_sha256(parent)
manifest["parent_packet_sha256"] = parent_packet_sha256
child_payload_sha256 = canonical_sha256(payload)
child_id = str(
uuid.uuid5(
uuid.NAMESPACE_URL,
f"livingip:normalized-strict-child:{parent_id}:{proposal_type}:{child_payload_sha256}",
)
)
source_ref = f"normalized:{parent_id}:{child_payload_sha256[:16]}"
child = {
"id": child_id,
"proposal_type": proposal_type,
"status": "pending_review",
"proposed_by_handle": parent.get("proposed_by_handle"),
"proposed_by_agent_id": parent.get("proposed_by_agent_id"),
"channel": parent.get("channel") or "clone_composition",
"source_ref": source_ref,
"rationale": f"Strict normalized child of rich source packet {parent_id}.",
"payload": payload,
"parent_packet_sha256": parent_packet_sha256,
"payload_sha256": child_payload_sha256,
}
validation_snapshot = {
**child,
"status": "approved",
"reviewed_by_handle": "normalization-preflight",
"reviewed_by_agent_id": None,
"reviewed_at": "2000-01-01T00:00:00+00:00",
"review_note": "Build-only validation of the exact normalized payload.",
}
ap.build_apply_sql(validation_snapshot, ap.SERVICE_AGENT_HANDLE)
return child
def build_stage_sql(child: dict[str, Any]) -> str:
proposal_id = ap.sql_literal(_uuid(child.get("id"), "child proposal id"))
proposal_type = ap.sql_literal(str(child["proposal_type"]))
proposed_by_handle = ap.sql_literal(str(child.get("proposed_by_handle") or ""))
proposed_by_agent_id = child.get("proposed_by_agent_id")
agent_sql = (
f"{ap.sql_literal(_uuid(proposed_by_agent_id, 'proposed_by_agent_id'))}::uuid"
if proposed_by_agent_id
else "null::uuid"
)
channel = ap.sql_literal(str(child["channel"]))
source_ref = ap.sql_literal(str(child["source_ref"]))
rationale = ap.sql_literal(str(child["rationale"]))
payload = ap.sql_literal(json.dumps(child["payload"], sort_keys=True))
return f"""\\set ON_ERROR_STOP on
begin;
create temporary table normalized_stage_result (created boolean not null) on commit drop;
with inserted as (
insert into kb_stage.kb_proposals
(id, proposal_type, status, proposed_by_handle, proposed_by_agent_id,
channel, source_ref, rationale, payload)
values
({proposal_id}::uuid, {proposal_type}, 'pending_review', nullif({proposed_by_handle}, ''),
{agent_sql}, {channel}, {source_ref}, {rationale}, {payload}::jsonb)
on conflict (id) do nothing
returning id
)
insert into normalized_stage_result(created)
select exists(select 1 from inserted);
do $normalized_stage$
declare
marker_count integer;
exact_count integer;
begin
select count(*) into marker_count
from kb_stage.kb_proposals
where source_ref = {source_ref};
select count(*) into exact_count
from kb_stage.kb_proposals
where id = {proposal_id}::uuid
and proposal_type = {proposal_type}
and status = 'pending_review'
and proposed_by_handle is not distinct from nullif({proposed_by_handle}, '')
and proposed_by_agent_id is not distinct from {agent_sql}
and channel = {channel}
and source_ref = {source_ref}
and rationale = {rationale}
and payload = {payload}::jsonb
and reviewed_by_handle is null
and reviewed_by_agent_id is null
and reviewed_at is null
and review_note is null
and applied_by_handle is null
and applied_by_agent_id is null
and applied_at is null;
if marker_count <> 1 or exact_count <> 1 then
raise exception 'normalized stage validation failed: marker_count=%, exact_count=%',
marker_count, exact_count;
end if;
end
$normalized_stage$;
select jsonb_build_object(
'created', (select created from normalized_stage_result),
'proposal_id', id::text,
'proposal_type', proposal_type,
'status', status,
'source_ref', source_ref,
'payload_sha256', {ap.sql_literal(str(child["payload_sha256"]))}
)::text
from kb_stage.kb_proposals
where id = {proposal_id}::uuid;
commit;
"""
def stage_normalized_child(container: str, database: str, parent: dict[str, Any]) -> dict[str, Any]:
target = bound.container_identity(container)
production = bound.container_identity(bound.PRODUCTION_CONTAINER)
bound.validate_disposable_target_identity(target, production)
if target["id"] == production["id"]:
raise bound.CheckpointError("normalized staging target resolves to production")
child = prepare_normalized_child(parent)
result = bound._psql_json(str(target["id"]), database, build_stage_sql(child))
if result.get("proposal_id") != child["id"] or result.get("status") != "pending_review":
raise bound.CheckpointError("normalized staging did not return the exact pending child")
return {"child": child, "database_receipt": result, "bound_container_id": str(target["id"])}
def parse_args(argv: list[str] | None = None) -> argparse.Namespace:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--container", required=True)
parser.add_argument("--db", required=True)
parser.add_argument("--proposal-file", required=True, type=Path)
return parser.parse_args(argv)
def main(argv: list[str] | None = None) -> int:
args = parse_args(argv)
try:
if args.container == bound.PRODUCTION_CONTAINER:
raise bound.CheckpointError("--container must be a disposable non-production target")
parent = json.loads(args.proposal_file.read_text(encoding="utf-8"))
if not isinstance(parent, dict):
raise bound.CheckpointError("--proposal-file must contain one JSON object")
result = stage_normalized_child(args.container, args.db, parent)
except (OSError, json.JSONDecodeError, bound.CheckpointError, SystemExit, ValueError) as exc:
print(json.dumps({"status": "rejected", "error": str(exc)}, sort_keys=True))
return 2
print(
json.dumps(
{
"status": "pending_review",
"proposal_id": result["child"]["id"],
"payload_sha256": result["child"]["payload_sha256"],
"created": result["database_receipt"].get("created"),
"bound_container_id": result["bound_container_id"],
},
sort_keys=True,
)
)
return 0
if __name__ == "__main__":
raise SystemExit(main())

View file

@ -0,0 +1,168 @@
from __future__ import annotations
import json
import sys
from pathlib import Path
from typing import Any
import pytest
REPO_ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(REPO_ROOT / "scripts"))
import bootstrap_clone_kb_gate as bootstrap # noqa: E402
def _identity(name: str, identifier: str, *, disposable: bool) -> dict[str, Any]:
return {
"id": identifier,
"name": name,
"image": "postgres:16",
"labels": (
{bootstrap.bound.DISPOSABLE_LABEL_KEY: bootstrap.bound.DISPOSABLE_LABEL_VALUE}
if disposable
else {}
),
"network_mode": "none" if disposable else "bridge",
"ports": {},
"mount_sources": [f"/{name}-data"],
"started_at": f"{name}-start",
}
def test_bootstrap_installs_gate_verifies_logins_and_writes_only_private_secret_files(
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
) -> None:
tmp_path.chmod(0o700)
target = _identity("clone", "clone-id", disposable=True)
production = _identity(bootstrap.bound.PRODUCTION_CONTAINER, "production-id", disposable=False)
monkeypatch.setattr(
bootstrap.bound,
"container_identity",
lambda name: dict(production if name == bootstrap.bound.PRODUCTION_CONTAINER else target),
)
monkeypatch.setattr(
bootstrap.bound,
"validate_disposable_target_identity",
lambda observed, live: {"all": observed["id"] != live["id"], "network_none": True},
)
passwords = iter(("review-test-secret", "apply-test-secret"))
monkeypatch.setattr(bootstrap.secrets, "token_urlsafe", lambda _length: next(passwords))
sql_calls: list[dict[str, Any]] = []
monkeypatch.setattr(
bootstrap,
"_psql",
lambda container, database, sql, **kwargs: sql_calls.append(
{"container": container, "database": database, "sql": sql, **kwargs}
),
)
login_calls: list[tuple[str, str, str, str]] = []
monkeypatch.setattr(
bootstrap,
"_verify_login",
lambda container, database, role, password: login_calls.append((container, database, role, password)),
)
manifest = {
"approval_table": "kb_stage.kb_proposal_approvals",
"review_principals_table": "kb_stage.kb_review_principals",
"review_role_exists": True,
"apply_role_exists": True,
}
monkeypatch.setattr(bootstrap.bound, "_psql_json", lambda *_args, **_kwargs: dict(manifest))
result = bootstrap.bootstrap("clone", "teleo", tmp_path)
assert result["status"] == "ready"
assert result["bound_container_id"] == "clone-id"
assert result["gate_manifest"] == manifest
assert result["secret_values_or_hashes_recorded"] is False
assert login_calls == [
("clone-id", "teleo", "kb_review", "review-test-secret"),
("clone-id", "teleo", "kb_apply", "apply-test-secret"),
]
assert len(sql_calls) == 3
assert sql_calls[-1]["secret_values"] == ("review-test-secret", "apply-test-secret")
review_file = tmp_path / "kb-review.env"
apply_file = tmp_path / "kb-apply.env"
assert review_file.read_text(encoding="utf-8") == "KB_REVIEW_PASSWORD=review-test-secret\n"
assert apply_file.read_text(encoding="utf-8") == "KB_APPLY_DB_PASSWORD=apply-test-secret\n"
assert review_file.stat().st_mode & 0o777 == 0o600
assert apply_file.stat().st_mode & 0o777 == 0o600
serialized = json.dumps(result, sort_keys=True)
assert "review-test-secret" not in serialized
assert "apply-test-secret" not in serialized
@pytest.mark.parametrize("mode", [0o755, 0o750])
def test_bootstrap_rejects_nonprivate_output_directory(
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
mode: int,
) -> None:
tmp_path.chmod(mode)
target = _identity("clone", "clone-id", disposable=True)
production = _identity(bootstrap.bound.PRODUCTION_CONTAINER, "production-id", disposable=False)
monkeypatch.setattr(
bootstrap.bound,
"container_identity",
lambda name: dict(production if name == bootstrap.bound.PRODUCTION_CONTAINER else target),
)
monkeypatch.setattr(bootstrap.bound, "validate_disposable_target_identity", lambda *_args: {"all": True})
with pytest.raises(bootstrap.bound.CheckpointError, match="private directory"):
bootstrap.bootstrap("clone", "teleo", tmp_path)
def test_bootstrap_refuses_existing_or_symlinked_secret_paths(
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
) -> None:
tmp_path.chmod(0o700)
target = _identity("clone", "clone-id", disposable=True)
production = _identity(bootstrap.bound.PRODUCTION_CONTAINER, "production-id", disposable=False)
monkeypatch.setattr(
bootstrap.bound,
"container_identity",
lambda name: dict(production if name == bootstrap.bound.PRODUCTION_CONTAINER else target),
)
monkeypatch.setattr(bootstrap.bound, "validate_disposable_target_identity", lambda *_args: {"all": True})
(tmp_path / "kb-review.env").symlink_to(tmp_path / "missing-target")
with pytest.raises(bootstrap.bound.CheckpointError, match="already exist"):
bootstrap.bootstrap("clone", "teleo", tmp_path)
def test_write_secret_is_exclusive_and_does_not_follow_symlinks(tmp_path: Path) -> None:
target = tmp_path / "target"
target.write_text("unchanged\n", encoding="utf-8")
link = tmp_path / "secret.env"
link.symlink_to(target)
with pytest.raises(FileExistsError):
bootstrap._write_secret(link, "SECRET", "value")
assert target.read_text(encoding="utf-8") == "unchanged\n"
def test_main_rejects_production_without_calling_bootstrap(
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
capsys: pytest.CaptureFixture[str],
) -> None:
called = False
def unexpected(*_args: Any, **_kwargs: Any) -> dict[str, Any]:
nonlocal called
called = True
return {}
monkeypatch.setattr(bootstrap, "bootstrap", unexpected)
result = bootstrap.main(
["--container", bootstrap.bound.PRODUCTION_CONTAINER, "--db", "teleo", "--output-dir", str(tmp_path)]
)
assert result == 2
assert called is False
output = json.loads(capsys.readouterr().out)
assert output["status"] == "rejected"
assert "non-production" in output["error"]

View file

@ -194,6 +194,71 @@ def test_rich_claim_graph_normalizes_to_one_atomic_approve_claim_bundle():
assert accounting["total_planned_source_rows"] == 3
def test_source_content_sha256_is_preserved_as_the_canonical_source_hash():
content_hash = "ab" * 32
preview = norm.normalize_proposal(
{
"id": "77777777-7777-4777-8777-777777777779",
"proposal_type": "attach_evidence",
"status": "pending_review",
"payload": {
"claim_candidates": [
{
"claim_key": "hash_bound_claim",
"text": "The claim is bound to source bytes.",
"evidence": [{"source_key": "hash_bound_source", "role": "grounds"}],
}
],
"source_candidates": [
{
"source_key": "hash_bound_source",
"source_type": "document",
"storage_path": "/evidence/source.txt",
"key_excerpt": "The claim is bound to source bytes.",
"content_sha256": content_hash.upper(),
}
],
},
}
)
assert preview["normalization_state"] == "strict_children_ready"
source_rows = preview["strict_child_proposals"][0]["payload"]["apply_payload"]["sources"]
source = next(row for row in source_rows if row["storage_path"] == "/evidence/source.txt")
assert source["hash"] == content_hash
assert content_hash.upper() in source["excerpt"]
def test_invalid_source_content_hash_blocks_rich_normalization():
preview = norm.normalize_proposal(
{
"id": "77777777-7777-4777-8777-777777777780",
"proposal_type": "attach_evidence",
"status": "pending_review",
"payload": {
"claim_candidates": [
{
"claim_key": "bad_hash_claim",
"text": "This must not normalize.",
"evidence": [{"source_key": "bad_hash_source"}],
}
],
"source_candidates": [
{
"source_key": "bad_hash_source",
"key_excerpt": "This must not normalize.",
"content_sha256": "not-a-sha256",
}
],
},
}
)
assert preview["normalization_state"] == "blocked_lossy_rich_normalization"
assert preview["strict_child_count"] == 0
assert any("invalid content_sha256" in item["reason"] for item in preview["blocked_fragments"])
def test_rich_normalization_blocks_malformed_or_duplicate_candidates():
proposal = {
"id": "77777777-7777-4777-8777-777777777777",
@ -247,17 +312,11 @@ def test_rich_normalization_retains_type_mapping_manifest_for_review():
)
child = preview["strict_child_proposals"][0]
manifest = child["payload"]["apply_payload"]["normalization_manifest"]
assert {
(item["claim_key"], item["from"], item["to"])
for item in manifest["claim_type_mappings"]
} == {
assert {(item["claim_key"], item["from"], item["to"]) for item in manifest["claim_type_mappings"]} == {
("governance_claim", "governance", "meta"),
("typed_claim", "normative/detail", "normative"),
}
assert {
(item["source_key"], item["from"], item["to"])
for item in manifest["source_type_mappings"]
} == {
assert {(item["source_key"], item["from"], item["to"]) for item in manifest["source_type_mappings"]} == {
("book_source", "book", "other"),
("web_source", "url", "article"),
("governance_claim__proposal_body", "proposal_body", "other"),

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,738 @@
from __future__ import annotations
import argparse
import asyncio
import copy
import json
import os
import subprocess
import sys
from pathlib import Path
from typing import Any
import pytest
REPO_ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(REPO_ROOT / "scripts"))
import run_leo_clone_composition_checkpoint as composition # noqa: E402
RUN_MARKER = "composition-unit-20260711"
CONVERSATION_MARKER = "composition-memory-unit-20260711"
AGENT_ID = "11111111-1111-4111-8111-111111111111"
def _packet(fixture: dict[str, Any]) -> dict[str, Any]:
return {
"claim_candidates": [
{
"claim_key": "guarded_apply_required",
"text": "Approval leaves canonical rows unchanged until guarded apply succeeds.",
"type": "structural",
"confidence": 0.98,
"evidence": [{"source_key": "operating_document", "role": "grounds"}],
"edges": [
{
"edge_type": "contradicts",
"target": "approval_is_canonical",
}
],
},
{
"claim_key": "approval_is_canonical",
"text": "Reviewer approval immediately makes a Leo proposal canonical.",
"type": "empirical",
"confidence": 0.4,
"evidence": [{"source_key": "contradictory_post", "role": "grounds"}],
},
],
"source_candidates": [
{
"source_key": "operating_document",
"source_type": "article",
"storage_path": fixture["document"]["storage_path"],
"url": None,
"title": fixture["document"]["title"],
"source_quality": "internal_operating_document",
"key_excerpt": fixture["document"]["text"],
"content_sha256": fixture["document"]["content_sha256"],
},
{
"source_key": "contradictory_post",
"source_type": "post",
"storage_path": None,
"url": fixture["post"]["url"],
"title": fixture["post"]["title"],
"source_quality": "unverified_social_post",
"key_excerpt": fixture["post"]["text"],
"content_sha256": fixture["post"]["content_sha256"],
},
],
"dedupe_conflict_assessment": {
"database_search_query": fixture["project"],
"potential_existing_claim_ids": [],
"conflicts": [
{
"from_claim_key": "guarded_apply_required",
"to_claim_key": "approval_is_canonical",
"relationship": "contradicts",
}
],
},
}
def test_fixture_and_prompts_keep_source_bytes_and_memory_boundaries_explicit() -> None:
fixture = composition.build_source_fixture(RUN_MARKER)
prompts = composition.build_prompts(fixture, CONVERSATION_MARKER)
assert fixture["project"] in fixture["document"]["text"]
assert fixture["project"] in fixture["post"]["text"]
assert fixture["document"]["content_sha256"] in prompts["extract"]
assert fixture["post"]["content_sha256"] in prompts["extract"]
assert CONVERSATION_MARKER in prompts["extract"]
assert CONVERSATION_MARKER not in prompts["reason"]
assert fixture["document"]["text"] not in prompts["reason"]
assert fixture["post"]["text"] not in prompts["reason"]
child = composition.normalized_stage.prepare_normalized_child(
composition.build_parent_packet(_packet(fixture), fixture, RUN_MARKER, AGENT_ID)
)
assert child["id"] not in prompts["reason"]
assert all(
row["id"] not in prompts["reason"]
for collection in ("claims", "sources")
for row in child["payload"]["apply_payload"][collection]
)
def test_valid_packet_is_hash_bound_evidenced_and_conflict_aware() -> None:
fixture = composition.build_source_fixture(RUN_MARKER)
validation = composition.validate_composition_packet(_packet(fixture), fixture)
assert validation["claim_count"] == 2
assert validation["source_count"] == 2
assert validation["evidence_source_keys"] == ["contradictory_post", "operating_document"]
assert validation["conflict_pairs"] == [["guarded_apply_required", "approval_is_canonical"]]
@pytest.mark.parametrize("mutation", ["hash", "excerpt", "evidence", "conflict"])
def test_packet_validation_fails_closed_on_provenance_or_reasoning_loss(mutation: str) -> None:
fixture = composition.build_source_fixture(RUN_MARKER)
packet = _packet(fixture)
if mutation == "hash":
packet["source_candidates"][0]["content_sha256"] = "0" * 64
elif mutation == "excerpt":
packet["source_candidates"][0]["key_excerpt"] = "hallucinated excerpt"
elif mutation == "evidence":
packet["claim_candidates"][0]["evidence"] = []
else:
packet["claim_candidates"][0]["edges"] = []
with pytest.raises(composition.bound.CheckpointError):
composition.validate_composition_packet(packet, fixture)
def test_packet_normalizes_to_deterministic_hash_bound_claim_graph() -> None:
fixture = composition.build_source_fixture(RUN_MARKER)
packet = _packet(fixture)
parent = composition.build_parent_packet(packet, fixture, RUN_MARKER, AGENT_ID)
child = composition.normalized_stage.prepare_normalized_child(parent)
payload = child["payload"]["apply_payload"]
assert parent["id"] == composition._stable_uuid(RUN_MARKER, "rich-source-packet")
assert len(payload["claims"]) == 2
assert len(payload["sources"]) == 4
assert len(payload["evidence"]) == 4
assert len(payload["edges"]) == 1
assert payload["edges"][0]["edge_type"] == "contradicts"
assert {fixture["document"]["content_sha256"], fixture["post"]["content_sha256"]} <= {
row["hash"] for row in payload["sources"]
}
assert composition.expected_table_deltas(child) == {
"public.claims": 2,
"public.sources": 4,
"public.claim_evidence": 4,
"public.claim_edges": 1,
"public.reasoning_tools": 0,
"kb_stage.kb_proposals": 1,
composition.lifecycle.APPROVAL_TABLE: 1,
}
def test_final_state_receipt_must_match_every_generated_row_id() -> None:
fixture = composition.build_source_fixture(RUN_MARKER)
child = composition.normalized_stage.prepare_normalized_child(
composition.build_parent_packet(_packet(fixture), fixture, RUN_MARKER, AGENT_ID)
)
payload = child["payload"]["apply_payload"]
edge = payload["edges"][0]
state = {
"phase": "final",
"conversation_marker": CONVERSATION_MARKER,
"proposal_id": child["id"],
"status": "applied",
"claim_ids": [row["id"] for row in payload["claims"]],
"source_ids": [row["id"] for row in payload["sources"]],
"conflict_edge": {
"from_claim": edge["from_claim"],
"to_claim": edge["to_claim"],
"edge_type": edge["edge_type"],
},
}
assert composition._state_matches(state, conversation_marker=CONVERSATION_MARKER, child=child) is True
marker_alias = copy.deepcopy(state)
marker_alias["marker"] = marker_alias.pop("conversation_marker")
assert composition._state_matches(marker_alias, conversation_marker=CONVERSATION_MARKER, child=child) is True
wrong = copy.deepcopy(state)
wrong["source_ids"] = wrong["source_ids"][:-1]
assert composition._state_matches(wrong, conversation_marker=CONVERSATION_MARKER, child=child) is False
def test_unrelated_guard_excludes_every_planned_row_and_is_read_only(
monkeypatch: pytest.MonkeyPatch,
) -> None:
fixture = composition.build_source_fixture(RUN_MARKER)
child = composition.normalized_stage.prepare_normalized_child(
composition.build_parent_packet(_packet(fixture), fixture, RUN_MARKER, AGENT_ID)
)
observed: dict[str, str] = {}
def fake_psql(_container: str, _database: str, sql: str) -> dict[str, Any]:
observed["sql"] = sql
return {table: {"count": 1, "rowset_md5": "same"} for table in composition.TABLES}
monkeypatch.setattr(composition.bound, "_psql_json", fake_psql)
snapshot = composition.unrelated_guard_snapshot("clone-id", "teleo", child)
lowered = observed["sql"].lower()
assert set(snapshot) == set(composition.TABLES)
assert "begin transaction read only" in lowered
assert child["id"] in observed["sql"]
for collection in ("claims", "sources"):
for row in child["payload"]["apply_payload"][collection]:
assert row["id"] in observed["sql"]
assert "insert into" not in lowered
assert "update " not in lowered
assert "delete from" not in lowered
def test_prefixed_receipts_are_exact_json_objects() -> None:
payload = {"claim_candidates": [], "source_candidates": []}
reply = "analysis\nCOMPOSITION_PACKET: " + json.dumps(payload, sort_keys=True)
assert composition.parse_prefixed_json(reply, composition.PACKET_PREFIX) == payload
with pytest.raises(composition.bound.CheckpointError, match="invalid"):
composition.parse_prefixed_json("COMPOSITION_PACKET: nope", composition.PACKET_PREFIX)
with pytest.raises(composition.bound.CheckpointError, match="did not contain"):
composition.parse_prefixed_json("no receipt", composition.PACKET_PREFIX)
def test_semantic_search_and_exact_read_helpers_require_database_commands_without_generated_ids() -> None:
claim_id = "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa"
proposal_id = "bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb"
project = "AURORA-TEST"
turn = {
"tool_invocations": [
{"argv": ["search", "approval apply canonical guarded operator review"]},
{"argv": ["search-proposals", project, "--status", "all", "--limit", "20"]},
{"argv": ["show-proposal", proposal_id]},
{"argv": ["show", claim_id]},
{"argv": ["evidence", claim_id, "--format", "json"]},
{"argv": ["edges", claim_id, "--format", "json"]},
]
}
assert composition._searched_semantically_without_ids(turn, [claim_id, proposal_id]) is True
assert composition._searched_composed_proposal(turn, project) is True
assert composition._read_exact_proposal(turn, proposal_id) is True
assert composition._read_claim(turn, claim_id) is True
assert composition._read_claim_evidence(turn, claim_id) is True
assert composition._read_claim_edges(turn, claim_id) is True
evidence_only = {"tool_invocations": [{"argv": ["evidence", claim_id, "--format", "json"]}]}
assert composition._read_claim_material(evidence_only, claim_id) is True
turn["tool_invocations"][0]["argv"][1] = f"show {claim_id}"
assert composition._searched_semantically_without_ids(turn, [claim_id, proposal_id]) is False
def test_cumulative_transcript_merge_accepts_only_receipted_truncated_unknown_results() -> None:
first = [
{"phase": "call", "tool_call_id": "call-1", "tool_name": "terminal"},
{
"phase": "result",
"tool_call_id": "call-1",
"tool_name": "terminal",
"content": '{"exit_code":0}',
"content_truncated": False,
},
]
second = [
*first,
{"phase": "call", "tool_call_id": "call-2", "tool_name": "terminal"},
{
"phase": "result",
"tool_call_id": "call-2",
"tool_name": "terminal",
"content": '{"output":"large',
"content_truncated": True,
},
]
gateways = [
{"transcript_tool_trace": {"events": first}},
{"transcript_tool_trace": {"events": second}},
]
merged = composition.merge_transcript_events(gateways)
consistency = composition.terminal_trace_consistency(
merged,
{
"invocation_count": 2,
"all_bound_to_supplied_target": True,
"all_completed_successfully": True,
},
)
assert len(merged) == 4
assert consistency["passes"] is True
assert consistency["unknown_result_call_ids"] == ["call-2"]
missing_receipt = [event for event in merged if event.get("tool_call_id") != "call-2" or event["phase"] != "result"]
assert composition.terminal_trace_consistency(
missing_receipt,
{
"invocation_count": 2,
"all_bound_to_supplied_target": True,
"all_completed_successfully": True,
},
)["passes"] is False
def test_source_contains_no_production_write_restart_or_delivery_adapter() -> None:
source = (REPO_ROOT / "scripts" / "run_leo_clone_composition_checkpoint.py").read_text(encoding="utf-8").lower()
assert "insert into public." not in source
assert "update public." not in source
assert "delete from public." not in source
assert "systemctl restart" not in source
assert "send_message(" not in source
def test_source_manifest_covers_the_full_composition_and_gate_dependency_closure() -> None:
paths = {item["repo_relative_path"] for item in composition.source_manifest()}
assert paths == {
"scripts/apply_proposal.py",
"scripts/apply_worker.py",
"scripts/approve_proposal.py",
"scripts/bootstrap_clone_kb_gate.py",
"scripts/kb_apply_prereqs.sql",
"scripts/kb_proposal_normalize.py",
"scripts/kb_proposal_review_packet.py",
"scripts/kb_rich_proposal_creation_plan.py",
"scripts/run_approve_claim_clone_canary.py",
"scripts/run_leo_clone_bound_handler_checkpoint.py",
"scripts/run_leo_clone_composition_checkpoint.py",
"scripts/run_leo_clone_lifecycle_checkpoint.py",
"scripts/stage_normalized_proposal.py",
}
def _args(tmp_path: Path) -> argparse.Namespace:
return argparse.Namespace(
container="composition-clone",
db="teleo",
output=tmp_path / "composition-report.json",
run_marker=RUN_MARKER,
conversation_marker=CONVERSATION_MARKER,
chat_id=composition.bound.DEFAULT_CHAT_ID,
user_id=composition.bound.DEFAULT_USER_ID,
user_name="composition unit",
turn_timeout=30,
review_secrets_file=str(tmp_path / "kb-review.env"),
apply_secrets_file=str(tmp_path / "kb-apply.env"),
copy_model_auth=True,
operator_review=True,
guarded_apply=True,
)
def _proposal(child: dict[str, Any], state: str) -> dict[str, Any]:
reviewed = state in {"approved", "applied"}
applied = state == "applied"
return {
"id": child["id"],
"proposal_type": child["proposal_type"],
"status": state,
"channel": child["channel"],
"source_ref": child["source_ref"],
"rationale": child["rationale"],
"payload": child["payload"],
"reviewed_by_handle": composition.lifecycle.REVIEWER_HANDLE if reviewed else None,
"reviewed_by_agent_id": AGENT_ID if reviewed else None,
"reviewed_at": "2026-07-11T10:00:00+00:00" if reviewed else None,
"review_note": composition.lifecycle.REVIEW_NOTE if reviewed else None,
"applied_by_handle": composition.lifecycle.APPLIED_BY_HANDLE if applied else None,
"applied_by_agent_id": AGENT_ID if applied else None,
"applied_at": "2026-07-11T10:01:00+00:00" if applied else None,
}
def _approval_snapshot(proposal: dict[str, Any]) -> dict[str, Any]:
return {
"proposal_id": proposal["id"],
"proposal_type": proposal["proposal_type"],
"payload": proposal["payload"],
"reviewed_by_handle": proposal["reviewed_by_handle"],
"reviewed_by_agent_id": proposal["reviewed_by_agent_id"],
"reviewed_at": proposal["reviewed_at"],
"review_note": proposal["review_note"],
}
def test_mocked_full_composition_checkpoint_passes_with_exact_rows_and_cleanup(
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
) -> None:
args = _args(tmp_path)
Path(args.review_secrets_file).write_text("KB_REVIEW_PASSWORD=test\n", encoding="utf-8")
Path(args.apply_secrets_file).write_text("KB_APPLY_PASSWORD=test\n", encoding="utf-8")
fixture = composition.build_source_fixture(args.run_marker)
packet = _packet(fixture)
parent = composition.build_parent_packet(packet, fixture, args.run_marker, AGENT_ID)
child = composition.normalized_stage.prepare_normalized_child(parent)
expected_rows = composition.guarded._expected_bundle_rows(child["payload"]["apply_payload"])
empty_rows = {key: [] for key in expected_rows}
base_counts = {table: 10 for table in composition.TABLES}
final_counts = {
table: base_counts[table] + composition.expected_table_deltas(child)[table] for table in composition.TABLES
}
base_snapshot = {
"counts": base_counts,
"rowset_md5": {table: f"base-{table}" for table in composition.TABLES},
}
state: dict[str, Any] = {"phase": "initial", "invocations": [], "transcript_events": []}
target_identity = {
"id": "composition-clone-id",
"name": args.container,
"image": "postgres",
"labels": {
composition.bound.DISPOSABLE_LABEL_KEY: composition.bound.DISPOSABLE_LABEL_VALUE,
},
"network_mode": "none",
"ports": {},
"mount_sources": ["/composition-clone-data"],
"started_at": "clone-start",
}
production_identity = {
"id": "production-id",
"name": composition.bound.PRODUCTION_CONTAINER,
"image": "postgres",
"labels": {},
"network_mode": "bridge",
"ports": {},
"mount_sources": ["/production-data"],
"started_at": "production-start",
}
monkeypatch.setattr(
composition.lifecycle,
"assert_disposable_target",
lambda _container, _database: (dict(target_identity), dict(production_identity)),
)
monkeypatch.setattr(
composition.bound,
"container_identity",
lambda name: dict(production_identity if name == composition.bound.PRODUCTION_CONTAINER else target_identity),
)
def guard_snapshot(container: str, _database: str, **_kwargs: Any) -> dict[str, Any]:
snapshot = json.loads(json.dumps(base_snapshot))
production = container == composition.bound.PRODUCTION_CONTAINER
snapshot["database_identity"] = {
"current_database": args.db,
"system_identifier": "production-system" if production else "clone-system",
"transaction_read_only": "off",
}
if not production and state["phase"] == "applied":
snapshot["counts"] = dict(final_counts)
for table, delta in composition.expected_table_deltas(child).items():
if delta:
snapshot["rowset_md5"][table] = "changed"
return snapshot
monkeypatch.setattr(composition.bound, "database_guard_snapshot", guard_snapshot)
clone_gate = {
"approval_table": composition.lifecycle.APPROVAL_TABLE,
"review_principals_table": "kb_stage.kb_review_principals",
"review_role_exists": True,
"apply_role_exists": True,
"approve_function": "kb_stage.approve_strict_proposal",
"assert_function": "kb_stage.assert_approved_proposal",
"finish_function": "kb_stage.finish_approved_proposal",
}
production_gate = {
"approval_table": None,
"review_principals_table": None,
"review_role_exists": False,
"apply_role_exists": True,
"approve_function": None,
"assert_function": None,
"finish_function": None,
}
monkeypatch.setattr(
composition,
"gate_schema_manifest",
lambda container, _database: (
dict(production_gate) if container == composition.bound.PRODUCTION_CONTAINER else dict(clone_gate)
),
)
monkeypatch.setattr(
composition.bound,
"service_state",
lambda: {"returncode": 0, "ActiveState": "active", "MainPID": "123", "NRestarts": "0"},
)
monkeypatch.setattr(
composition.bound,
"bridge_hashes",
lambda: {"wrapper": "live-wrapper", "bridge_skill": "live-skill"},
)
runtime_root = tmp_path / "composition-runtime"
runtime_profile = runtime_root / "profile"
def copy_profile(**_kwargs: Any) -> tuple[Path, dict[str, Any]]:
runtime_profile.mkdir(parents=True)
composition.bound.register_temp_root(runtime_root)
return runtime_profile, {"passes": True, "secret_contents_read_or_recorded": False}
monkeypatch.setattr(composition.bound, "copy_profile", copy_profile)
monkeypatch.setattr(
composition.bound,
"copy_ephemeral_model_auth",
lambda _profile: {"mode": "ephemeral_file_copy", "source_contents_recorded": False},
)
monkeypatch.setattr(
composition.bound,
"load_ephemeral_provider_environment",
lambda _profile: ({}, {"bound_credential_count": 0, "credential_contents_recorded": False}),
)
tool_log = runtime_profile / "tool-log.jsonl"
monkeypatch.setattr(
composition,
"patch_composition_bridge",
lambda _profile, **_kwargs: {
"tool_log_path": str(tool_log),
"run_nonce": "composition-run-nonce",
"wrapper_path": str(runtime_profile / "bin" / "teleo-kb"),
"bridge_skill_path": str(runtime_profile / "skills" / "teleo-kb-bridge" / "SKILL.md"),
},
)
monkeypatch.setattr(composition, "resolve_leo_agent_id", lambda _container, _database: AGENT_ID)
monkeypatch.setattr(
composition.normalized_stage,
"stage_normalized_child",
lambda _container, _database, _parent: (
state.update(phase="pending_review")
or {
"child": child,
"bound_container_id": target_identity["id"],
"database_receipt": {
"created": True,
"proposal_id": child["id"],
"proposal_type": child["proposal_type"],
"status": "pending_review",
"source_ref": child["source_ref"],
"payload_sha256": child["payload_sha256"],
},
}
),
)
monkeypatch.setattr(
composition.guarded,
"_exact_bundle_readback",
lambda _container, _database, _payload: expected_rows if state["phase"] == "applied" else empty_rows,
)
monkeypatch.setattr(
composition.guarded,
"_proposal_readback",
lambda _container, _database, _proposal_id: (
None if state["phase"] == "initial" else _proposal(child, state["phase"])
),
)
monkeypatch.setattr(
composition.guarded,
"_approval_snapshot_readback",
lambda _container, _database, _proposal_id: _approval_snapshot(_proposal(child, "approved")),
)
unrelated = {table: {"count": 10, "rowset_md5": f"unrelated-{table}"} for table in composition.TABLES}
monkeypatch.setattr(
composition,
"unrelated_guard_snapshot",
lambda _container, _database, _child: json.loads(json.dumps(unrelated)),
)
def review(
_args: argparse.Namespace, _proposal_id: str
) -> tuple[subprocess.CompletedProcess[str], subprocess.CompletedProcess[str]]:
state["phase"] = "approved"
return (
subprocess.CompletedProcess([], 0, "select kb_stage.approve_strict_proposal(...);", ""),
subprocess.CompletedProcess([], 0, "approved", ""),
)
def apply(
_args: argparse.Namespace, _proposal_id: str
) -> tuple[subprocess.CompletedProcess[str], subprocess.CompletedProcess[str]]:
state["phase"] = "applied"
return (
subprocess.CompletedProcess(
[],
0,
"select kb_stage.assert_approved_proposal(...); select kb_stage.finish_approved_proposal(...);",
"",
),
subprocess.CompletedProcess([], 0, "applied", ""),
)
monkeypatch.setattr(composition.lifecycle, "run_operator_review", review)
monkeypatch.setattr(composition.lifecycle, "run_guarded_apply", apply)
async def run_turn(
_args: argparse.Namespace,
_profile: Path,
*,
phase: str,
prompt: str,
provider_environment: dict[str, str],
tool_log: Path,
) -> dict[str, Any]:
assert provider_environment == {}
assert tool_log == runtime_profile / "tool-log.jsonl"
if phase == "T1_extract_sources":
reply = "Extracted two conflicting claims.\nCOMPOSITION_PACKET: " + json.dumps(
packet, sort_keys=True, separators=(",", ":")
)
argvs = [["context", packet["dedupe_conflict_assessment"]["database_search_query"]]]
pid = 101
else:
payload = child["payload"]["apply_payload"]
edge = payload["edges"][0]
receipt = {
"phase": "final",
"conversation_marker": args.conversation_marker,
"proposal_id": child["id"],
"status": "applied",
"claim_ids": [row["id"] for row in payload["claims"]],
"source_ids": [row["id"] for row in payload["sources"]],
"conflict_edge": {
"from_claim": edge["from_claim"],
"to_claim": edge["to_claim"],
"edge_type": edge["edge_type"],
},
}
reply = (
"Approval was not canonical; guarded apply made it applied and canonical. The sources conflict. "
f"I recall {args.conversation_marker}.\nCOMPOSITION_STATE: "
+ json.dumps(receipt, sort_keys=True, separators=(",", ":"))
)
argvs = [
["search", "approval apply canonical guarded operator review"],
["search-proposals", fixture["project"], "--status", "all", "--limit", "20"],
["show-proposal", child["id"]],
*[["show", row["id"]] for row in payload["claims"]],
*[["evidence", row["id"], "--format", "json"] for row in payload["claims"]],
["edges", payload["claims"][0]["id"], "--format", "json"],
]
pid = 202
invocations = [
{
"invocation_id": f"{phase}-{index}",
"container": target_identity["id"],
"database": args.db,
"argv": argv,
"returncode": 0,
}
for index, argv in enumerate(argvs)
]
state["invocations"].extend(invocations)
current_events = [
event
for invocation in invocations
for event in (
{
"phase": "call",
"tool_name": "terminal",
"tool_call_id": invocation["invocation_id"],
},
{
"phase": "result",
"tool_name": "terminal",
"tool_call_id": invocation["invocation_id"],
"content": '{"exit_code":0}',
"content_truncated": False,
},
)
]
state["transcript_events"].extend(current_events)
events = copy.deepcopy(state["transcript_events"])
return {
"phase": phase,
"prompt": prompt,
"prompt_sha256": "prompt-hash",
"tool_invocations": invocations,
"gateway": {
"authorized": True,
"handler_invoked": True,
"posted_to_telegram": False,
"model_free_fallback_used": False,
"reply": reply,
"session_key": "telegram:composition",
"transcript_tool_trace": {
"session_id": "persisted-composition-session",
"events": events,
},
"tool_surface": {
"allowed_tools": ["skills_list", "skill_view", "terminal"],
"actual_registry_tools": ["skills_list", "skill_view", "terminal"],
"gateway_adapters_verified_mapping": True,
"gateway_adapter_count": 0,
"send_message_tool_enabled": False,
"terminal_restricted_to_clone_wrapper": True,
"terminal_subprocess_inherits_provider_credentials": False,
},
"child_process": {
"pid": pid,
"readiness_verified": True,
"timed_out": False,
"exitcode": 0,
"result_transport": "private_temp_file",
"termination": {"attempted": False},
"alive_after_readback": False,
"process_group_alive_after_readback": False,
},
},
}
monkeypatch.setattr(composition, "run_turn", run_turn)
monkeypatch.setattr(
composition.bound,
"read_tool_proof",
lambda _path, _container, _database, **_kwargs: {
"invocations": list(state["invocations"]),
"invocation_count": len(state["invocations"]),
"all_bound_to_supplied_target": bool(state["invocations"]),
"all_completed_successfully": bool(state["invocations"]),
},
)
report = asyncio.run(composition.run_checkpoint(args))
assert report["status"] == "pass", {
"errors": report["errors"],
"failed_checks": [name for name, passed in report["checks"].items() if not passed],
}
assert all(report["checks"].values())
assert report["final_canonical_rows"] == expected_rows
assert report["checks"]["target_table_deltas_exact"] is True
assert report["cleanup"]["active_gateway_registry_clear"] is True
assert not os.path.lexists(runtime_root)
retained = json.loads(args.output.read_text(encoding="utf-8"))
assert retained["status"] == "pass"
assert args.output.stat().st_mode & 0o777 == 0o600

View file

@ -0,0 +1,823 @@
"""Focused tests for the no-send clone lifecycle checkpoint."""
from __future__ import annotations
import argparse
import asyncio
import json
import os
import subprocess
import sys
import uuid
from pathlib import Path
from typing import Any
import pytest
REPO_ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(REPO_ROOT / "scripts"))
import apply_proposal as ap # noqa: E402
import run_leo_clone_lifecycle_checkpoint as lifecycle # noqa: E402
RUN_MARKER = "leo-lifecycle-unit-20260710"
CONVERSATION_MARKER = "conversation-memory-unit-20260710"
@pytest.fixture(autouse=True)
def _clear_bound_registries() -> None:
lifecycle.bound._ACTIVE_TEMP_ROOTS.clear()
lifecycle.bound._ACTIVE_GATEWAY_CHILDREN.clear()
lifecycle.bound._LAST_TERMINATION_SIGNAL = None
yield
lifecycle.bound._ACTIVE_TEMP_ROOTS.clear()
lifecycle.bound._ACTIVE_GATEWAY_CHILDREN.clear()
lifecycle.bound._LAST_TERMINATION_SIGNAL = None
def _args(tmp_path: Path, **overrides: Any) -> argparse.Namespace:
review_secret = tmp_path / "kb-review.env"
apply_secret = tmp_path / "kb-apply.env"
review_secret.write_text("KB_REVIEW_PASSWORD=private\n", encoding="utf-8")
apply_secret.write_text("KB_APPLY_PASSWORD=private\n", encoding="utf-8")
values: dict[str, Any] = {
"container": "teleo-lifecycle-clone",
"db": "teleo",
"output": tmp_path / "receipt.json",
"run_marker": RUN_MARKER,
"conversation_marker": CONVERSATION_MARKER,
"chat_id": "clone-chat",
"user_id": "clone-user",
"user_name": "clone operator",
"turn_timeout": 30,
"review_secrets_file": str(review_secret),
"apply_secrets_file": str(apply_secret),
"copy_model_auth": True,
"operator_review": True,
"guarded_apply": True,
}
values.update(overrides)
return argparse.Namespace(**values)
def _proposal(fixture: dict[str, Any], state: str) -> dict[str, Any]:
row = {
"id": fixture["proposal_id"],
"proposal_type": "approve_claim",
"status": state,
"channel": "clone_lifecycle",
"source_ref": fixture["source_ref"],
"rationale": fixture["rationale"],
"payload": fixture["proposal_payload"],
"reviewed_by_handle": None,
"reviewed_by_agent_id": None,
"reviewed_at": None,
"review_note": None,
"applied_by_handle": None,
"applied_by_agent_id": None,
"applied_at": None,
}
if state in {"approved", "applied"}:
row.update(
{
"reviewed_by_handle": lifecycle.REVIEWER_HANDLE,
"reviewed_at": "2026-07-10T10:00:00+00:00",
"review_note": lifecycle.REVIEW_NOTE,
}
)
if state == "applied":
row.update(
{
"applied_by_handle": lifecycle.APPLIED_BY_HANDLE,
"applied_by_agent_id": "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa",
"applied_at": "2026-07-10T10:01:00+00:00",
}
)
return row
def _approval_snapshot(proposal: dict[str, Any]) -> dict[str, Any]:
return {
"proposal_id": proposal["id"],
"proposal_type": proposal["proposal_type"],
"payload": proposal["payload"],
"reviewed_by_handle": proposal["reviewed_by_handle"],
"reviewed_by_agent_id": proposal["reviewed_by_agent_id"],
"reviewed_by_db_role": "kb_review",
"reviewed_at": proposal["reviewed_at"],
"review_note": proposal["review_note"],
}
def test_fixture_is_deterministic_narrow_and_accepted_by_guarded_apply() -> None:
first = lifecycle.build_lifecycle_fixture(RUN_MARKER)
second = lifecycle.build_lifecycle_fixture(RUN_MARKER)
assert first == second
assert len(first["apply_payload"]["claims"]) == 1
assert len(first["apply_payload"]["sources"]) == 1
assert len(first["apply_payload"]["evidence"]) == 1
assert first["apply_payload"]["edges"] == []
assert first["apply_payload"]["reasoning_tools"] == []
assert first["proposal_payload"]["lifecycle_checkpoint"]["model_authority"] == "stage_only"
for key in ("proposal_id", "claim_id", "source_id"):
uuid.UUID(first[key])
approved = _proposal(first, "approved")
sql = ap.build_apply_sql(approved, lifecycle.APPLIED_BY_HANDLE)
assert "kb_stage.assert_approved_proposal" in sql
assert "kb_stage.finish_approved_proposal" in sql
assert first["claim_id"] in sql
assert first["source_id"] in sql
def test_stage_sql_is_idempotent_and_cannot_write_canonical_tables() -> None:
fixture = lifecycle.build_lifecycle_fixture(RUN_MARKER)
sql = lifecycle.build_stage_sql(fixture)
lowered = sql.lower()
assert "insert into kb_stage.kb_proposals" in lowered
assert "on conflict (id) do nothing" in lowered
assert "pending_review" in sql
assert fixture["source_ref"] in sql
assert "lifecycle stage validation failed" in lowered
assert "marker_count <> 1 or exact_count <> 1" in lowered
assert lowered.index("raise exception") < lowered.index("commit;")
assert "insert into public." not in lowered
assert "update public." not in lowered
assert "delete from public." not in lowered
def test_unrelated_row_guard_is_read_only_and_excludes_only_deterministic_rows(
monkeypatch: pytest.MonkeyPatch,
) -> None:
fixture = lifecycle.build_lifecycle_fixture(RUN_MARKER)
captured: dict[str, str] = {}
def fake_psql(_container: str, _database: str, sql: str) -> dict[str, Any]:
captured["sql"] = sql
return {table: {"count": 1, "rowset_md5": "same"} for table in lifecycle.LIFECYCLE_TABLES}
monkeypatch.setattr(lifecycle.bound, "_psql_json", fake_psql)
snapshot = lifecycle.target_unrelated_guard_snapshot("clone", "teleo", fixture)
sql = captured["sql"].lower()
assert set(snapshot) == set(lifecycle.LIFECYCLE_TABLES)
assert "begin transaction read only" in sql
assert fixture["proposal_id"] in sql
assert fixture["claim_id"] in sql
assert fixture["source_id"] in sql
assert lifecycle.APPROVAL_TABLE in sql
assert "insert into" not in sql
assert "update " not in sql
assert "delete from" not in sql
def test_stage_replay_returns_the_same_pending_proposal(
monkeypatch: pytest.MonkeyPatch,
) -> None:
fixture = lifecycle.build_lifecycle_fixture(RUN_MARKER)
pending = _proposal(fixture, "pending_review")
monkeypatch.setattr(
lifecycle,
"assert_disposable_target",
lambda _container, _database: ({"id": "clone"}, {"id": "production"}),
)
monkeypatch.setattr(
lifecycle.bound,
"_psql_json",
lambda _container, _database, _sql: {
"created": False,
"marker_match_count": 1,
"proposal": pending,
},
)
replay = lifecycle.stage_lifecycle_proposal("clone", "teleo", fixture)
assert replay["created"] is False
assert replay["idempotent_existing"] is True
assert replay["proposal"] == pending
assert replay["canonical_row_ids"]["public.claims"] == [fixture["claim_id"]]
def test_staging_wrapper_hard_binds_target_and_exposes_no_review_apply_or_send(tmp_path: Path) -> None:
wrapper = lifecycle.build_lifecycle_wrapper(
kb_tool=tmp_path / "kb_tool.py",
tool_log=tmp_path / "tool.jsonl",
container="clone-container",
database="teleo_clone",
run_marker=RUN_MARKER,
run_nonce="unit-test-nonce",
)
assert "TARGET_CONTAINER=clone-container" in wrapper
assert "TARGET_DATABASE=teleo_clone" in wrapper
assert wrapper.startswith("#!/bin/bash\n")
assert f"DOCKER={lifecycle.bound.SYSTEM_DOCKER}" in wrapper
assert f"export PATH={lifecycle.bound.SYSTEM_EXEC_PATH}" in wrapper
assert "/usr/bin/env bash" not in wrapper
assert "$(docker " not in wrapper
assert lifecycle.INTERNAL_STAGE_COMMAND in wrapper
assert f'${{1:-}}" == {lifecycle.STAGE_COMMAND}' in wrapper
assert "accepts no model-supplied arguments" in wrapper
assert "approve_proposal.py" not in wrapper
assert "apply_proposal.py" not in wrapper
assert "send_message" not in wrapper
def test_lifecycle_surface_temporarily_adds_only_the_stage_verb(
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
) -> None:
original_commands = lifecycle.bound.READ_ONLY_BRIDGE_COMMANDS
def fake_install(_profile: Path) -> dict[str, Any]:
return {
"allowed_tools": ["skills_list", "skill_view", "terminal"],
"read_only_bridge_commands": sorted(lifecycle.bound.READ_ONLY_BRIDGE_COMMANDS),
"send_message_tool_enabled": False,
}
monkeypatch.setattr(lifecycle.bound, "install_checkpoint_tool_surface", fake_install)
with lifecycle.lifecycle_gateway_surface(allow_staging=True):
assert lifecycle.STAGE_COMMAND in lifecycle.bound.READ_ONLY_BRIDGE_COMMANDS
surface = lifecycle.bound.install_checkpoint_tool_surface(tmp_path)
assert surface["staging_bridge_commands"] == [lifecycle.STAGE_COMMAND]
assert surface["model_can_stage_pending_proposal"] is True
assert surface["model_can_approve_or_apply"] is False
assert surface["operator_review_apply_tools_exposed"] is False
assert "send_message" not in surface["allowed_tools"]
assert original_commands == lifecycle.bound.READ_ONLY_BRIDGE_COMMANDS
assert lifecycle.STAGE_COMMAND not in lifecycle.bound.READ_ONLY_BRIDGE_COMMANDS
with lifecycle.lifecycle_gateway_surface(allow_staging=False):
surface = lifecycle.bound.install_checkpoint_tool_surface(tmp_path)
assert surface["staging_bridge_commands"] == []
assert surface["model_can_stage_pending_proposal"] is False
with pytest.raises(lifecycle.bound.CheckpointError):
lifecycle.bound.validate_read_only_bridge_argv([lifecycle.STAGE_COMMAND])
def test_prompts_keep_conversation_marker_out_of_mutation_and_recall_prompt() -> None:
fixture = lifecycle.build_lifecycle_fixture(RUN_MARKER)
prompts = lifecycle.lifecycle_prompts(fixture, CONVERSATION_MARKER)
assert CONVERSATION_MARKER in prompts["T1_conversation_marker"]
assert CONVERSATION_MARKER not in prompts["T2_stage_pending_proposal"]
assert CONVERSATION_MARKER not in prompts["T5_recall_and_applied_readback"]
assert CONVERSATION_MARKER not in json.dumps(fixture, sort_keys=True)
assert RUN_MARKER in prompts["T2_stage_pending_proposal"]
assert RUN_MARKER not in prompts["T5_recall_and_applied_readback"]
def test_lifecycle_state_receipt_is_exact_and_rejects_extra_or_contradictory_state() -> None:
expected = {
"phase": "T2",
"proposal_id": "537c3087-b795-5382-8335-e0928d1ac96c",
"status": "pending_review",
"canonical": False,
}
reply = "Proposal staged and not canonical.\nLIFECYCLE_STATE: " + json.dumps(expected)
assert lifecycle.parse_lifecycle_state(reply) == {"valid": True, "value": expected}
assert lifecycle.lifecycle_state_matches(reply, expected) is True
assert lifecycle.lifecycle_state_matches(reply.replace("}", ', "unverified": true}', 1), expected) is False
assert lifecycle._reply_has_phase_contradiction("T2", reply) is False
assert (
lifecycle._reply_has_phase_contradiction("T2", "The proposal is now canonical.\n" + reply.splitlines()[-1])
is True
)
assert lifecycle.parse_lifecycle_state("LIFECYCLE_STATE: not-json")["valid"] is False
def test_phase_row_read_requires_the_exact_full_uuid() -> None:
turn = {"tool_invocations": [{"argv": ["show-proposal", "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa"]}]}
assert lifecycle._phase_has_bound_read(turn, "show-proposal", "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa") is True
assert lifecycle._phase_has_bound_read(turn, "show-proposal", "bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb") is False
def test_validation_requires_explicit_clone_review_apply_and_auth(tmp_path: Path) -> None:
lifecycle.validate_args(_args(tmp_path))
with pytest.raises(lifecycle.bound.CheckpointError, match="production"):
lifecycle.validate_args(_args(tmp_path, container=lifecycle.bound.PRODUCTION_CONTAINER))
with pytest.raises(lifecycle.bound.CheckpointError, match="operator-review"):
lifecycle.validate_args(_args(tmp_path, operator_review=False))
with pytest.raises(lifecycle.bound.CheckpointError, match="guarded-apply"):
lifecycle.validate_args(_args(tmp_path, guarded_apply=False))
with pytest.raises(lifecycle.bound.CheckpointError, match="copy-model-auth"):
lifecycle.validate_args(_args(tmp_path, copy_model_auth=False))
def test_validation_rejects_receipt_inside_live_profile(
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
) -> None:
live = tmp_path / "live-profile"
live.mkdir()
monkeypatch.setattr(lifecycle.bound, "LIVE_PROFILE", live)
with pytest.raises(lifecycle.bound.CheckpointError, match="outside the live leoclean profile"):
lifecycle.validate_args(_args(tmp_path, output=live / "receipt.json"))
def test_disposable_identity_rejects_an_alias_of_production(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.setattr(
lifecycle.bound,
"container_identity",
lambda name: {"id": "same-id", "name": name, "image": "postgres"},
)
with pytest.raises(lifecycle.bound.CheckpointError, match="production container identity"):
lifecycle.assert_disposable_target("alias-container", "teleo")
def _install_fake_runtime(
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
args: argparse.Namespace,
) -> tuple[dict[str, str], dict[str, Any]]:
fixture = lifecycle.build_lifecycle_fixture(args.run_marker)
expected_rows = lifecycle.guarded._expected_bundle_rows(fixture["apply_payload"])
empty_rows = {key: [] for key in expected_rows}
base_counts = {
"public.claims": 100,
"public.sources": 200,
"public.claim_evidence": 300,
"public.claim_edges": 400,
"public.reasoning_tools": 5,
"kb_stage.kb_proposals": 9,
lifecycle.APPROVAL_TABLE: 4,
}
stage_counts = dict(base_counts)
stage_counts["kb_stage.kb_proposals"] += 1
approved_counts = dict(stage_counts)
approved_counts[lifecycle.APPROVAL_TABLE] += 1
final_counts = dict(approved_counts)
final_counts["public.claims"] += 1
final_counts["public.sources"] += 1
final_counts["public.claim_evidence"] += 1
state: dict[str, Any] = {"phase": "initial", "invocations": []}
base_snapshot = {
"counts": base_counts,
"rowset_md5": {table: f"production-{index}" for index, table in enumerate(base_counts)},
}
target_identity = {
"id": "clone-id",
"name": args.container,
"image": "postgres",
"labels": {
lifecycle.bound.DISPOSABLE_LABEL_KEY: lifecycle.bound.DISPOSABLE_LABEL_VALUE,
},
"network_mode": "none",
"ports": {},
"mount_sources": ["/clone-data"],
"started_at": "clone-start",
}
production_identity = {
"id": "production-id",
"name": lifecycle.bound.PRODUCTION_CONTAINER,
"image": "postgres",
"labels": {},
"network_mode": "bridge",
"ports": {},
"mount_sources": ["/production-data"],
"started_at": "production-start",
}
monkeypatch.setattr(
lifecycle,
"assert_disposable_target",
lambda _container, _database: (
dict(target_identity),
dict(production_identity),
),
)
monkeypatch.setattr(
lifecycle.bound,
"container_identity",
lambda container: dict(
production_identity if container == lifecycle.bound.PRODUCTION_CONTAINER else target_identity
),
)
def fake_guard_snapshot(container: str, _database: str, **_kwargs: Any) -> dict[str, Any]:
if container == lifecycle.bound.PRODUCTION_CONTAINER:
snapshot = json.loads(json.dumps(base_snapshot))
snapshot["database_identity"] = {
"current_database": args.db,
"system_identifier": "production-system",
"transaction_read_only": "off",
}
return snapshot
identity = {
"current_database": args.db,
"system_identifier": "clone-system",
"transaction_read_only": "off",
}
if state["phase"] == "applied":
snapshot = json.loads(json.dumps(base_snapshot))
snapshot["counts"] = dict(final_counts)
for table in (
"public.claims",
"public.sources",
"public.claim_evidence",
"kb_stage.kb_proposals",
lifecycle.APPROVAL_TABLE,
):
snapshot["rowset_md5"][table] = "changed"
snapshot["database_identity"] = identity
return snapshot
snapshot = json.loads(json.dumps(base_snapshot))
snapshot["database_identity"] = identity
return snapshot
monkeypatch.setattr(lifecycle.bound, "database_guard_snapshot", fake_guard_snapshot)
monkeypatch.setattr(
lifecycle.bound,
"service_state",
lambda: {"returncode": 0, "ActiveState": "active", "MainPID": "123", "NRestarts": "0"},
)
monkeypatch.setattr(
lifecycle.bound,
"bridge_hashes",
lambda: {"wrapper": "live-wrapper-hash", "bridge_skill": "live-skill-hash"},
)
monkeypatch.setattr(
lifecycle,
"target_unrelated_guard_snapshot",
lambda _container, _database, _fixture: {
table: {"count": base_counts[table], "rowset_md5": f"unrelated-{table}"}
for table in lifecycle.LIFECYCLE_TABLES
},
)
monkeypatch.setattr(lifecycle, "_source_manifest", lambda: [{"repo_relative_path": "stable", "sha256": "x"}])
def fake_proposal_readback(_container: str, _database: str, _proposal_id: str) -> dict[str, Any] | None:
if state["phase"] == "initial":
return None
return _proposal(fixture, state["phase"])
monkeypatch.setattr(lifecycle.guarded, "_proposal_readback", fake_proposal_readback)
monkeypatch.setattr(
lifecycle.guarded,
"_exact_bundle_readback",
lambda _container, _database, _payload: expected_rows if state["phase"] == "applied" else empty_rows,
)
monkeypatch.setattr(
lifecycle.guarded,
"_approval_snapshot_readback",
lambda _container, _database, _proposal_id: _approval_snapshot(_proposal(fixture, "approved")),
)
runtime_root = tmp_path / "private-runtime"
runtime_profile = runtime_root / "profile"
def fake_copy_profile(**_kwargs: Any) -> tuple[Path, dict[str, Any]]:
runtime_profile.mkdir(parents=True)
lifecycle.bound.register_temp_root(runtime_root)
return runtime_profile, {"passes": True, "secret_contents_read_or_recorded": False}
monkeypatch.setattr(lifecycle.bound, "copy_profile", fake_copy_profile)
monkeypatch.setattr(
lifecycle.bound,
"copy_ephemeral_model_auth",
lambda _profile: {
"mode": "ephemeral_file_copy",
"source_contents_recorded": False,
"source_fingerprint_recorded": False,
},
)
monkeypatch.setattr(
lifecycle.bound,
"load_ephemeral_provider_environment",
lambda _profile: ({}, {"bound_credential_count": 0, "credential_contents_recorded": False}),
)
tool_log = runtime_profile / "checkpoint-tool-calls.jsonl"
monkeypatch.setattr(
lifecycle,
"patch_lifecycle_bridge",
lambda _profile, **_kwargs: {
"wrapper_path": str(runtime_profile / "bin" / "teleo-kb"),
"bridge_skill_path": str(runtime_profile / "skills" / "teleo-kb-bridge" / "SKILL.md"),
"tool_log_path": str(tool_log),
"run_nonce": "lifecycle-test-nonce",
"model_can_approve": False,
"model_can_apply": False,
},
)
async def fake_turn(
_args: argparse.Namespace,
_profile: Path,
*,
phase: str,
prompt: str,
provider_environment: dict[str, str],
tool_log: Path,
) -> dict[str, Any]:
assert provider_environment == {}
assert tool_log == runtime_profile / "checkpoint-tool-calls.jsonl"
command_argvs: list[list[str]]
if phase == "T1_conversation_marker":
reply = (
f"I remember {args.conversation_marker}; conversation memory is not canonical Postgres.\n"
f'{lifecycle.LIFECYCLE_STATE_PREFIX} {{"phase":"T1","conversation_marker":'
f'"{args.conversation_marker}","kb_mutated":false}}'
)
command_argvs = []
elif phase == "T2_stage_pending_proposal":
state["phase"] = "pending_review"
reply = (
f"Staged proposal {fixture['proposal_id']} as pending_review, not canonical.\n"
f'{lifecycle.LIFECYCLE_STATE_PREFIX} {{"phase":"T2","proposal_id":"{fixture["proposal_id"]}",'
'"status":"pending_review","canonical":false}'
)
command_argvs = [
[lifecycle.STAGE_COMMAND],
["show-proposal", fixture["proposal_id"]],
]
elif phase == "T3_approved_unapplied_readback":
reply = (
f"Proposal {fixture['proposal_id']} is approved, applied_at NULL, and not canonical.\n"
f'{lifecycle.LIFECYCLE_STATE_PREFIX} {{"phase":"T3","proposal_id":"{fixture["proposal_id"]}",'
'"status":"approved","applied":false,"canonical":false}'
)
command_argvs = [["show-proposal", fixture["proposal_id"]]]
else:
reply = (
f"Marker {args.conversation_marker}; proposal {fixture['proposal_id']} is applied; "
f"canonical claim {fixture['claim_id']} and source {fixture['source_id']}.\n"
f'{lifecycle.LIFECYCLE_STATE_PREFIX} {{"phase":"T5","conversation_marker":'
f'"{args.conversation_marker}","proposal_id":"{fixture["proposal_id"]}","status":"applied",'
f'"claim_id":"{fixture["claim_id"]}","source_id":"{fixture["source_id"]}"}}'
)
command_argvs = [
["show-proposal", fixture["proposal_id"]],
["show", fixture["claim_id"]],
]
invocations = [
{
"invocation_id": f"{phase}-{index}",
"container": args.bound_container_id,
"database": args.db,
"argv": command_argv,
"returncode": 0,
}
for index, command_argv in enumerate(command_argvs)
]
state["invocations"].extend(invocations)
pid = {
"T1_conversation_marker": 101,
"T2_stage_pending_proposal": 102,
"T3_approved_unapplied_readback": 103,
"T5_recall_and_applied_readback": 105,
}[phase]
gateway = {
"authorized": True,
"handler_invoked": True,
"posted_to_telegram": False,
"model_free_fallback_used": False,
"reply": reply,
"session_key": "telegram:clone-chat",
"transcript_tool_trace": {
"session_id": "persisted-private-session",
"events": [
event
for item in invocations
for event in (
{"phase": "call", "tool_name": "terminal", "tool_call_id": item["invocation_id"]},
{
"phase": "result",
"tool_name": "terminal",
"tool_call_id": item["invocation_id"],
"content": '{"exit_code": 0}',
},
)
],
},
"tool_surface": {
"allowed_tools": ["skills_list", "skill_view", "terminal"],
"actual_registry_tools": ["skills_list", "skill_view", "terminal"],
"send_message_tool_enabled": False,
"gateway_adapters_verified_mapping": True,
"gateway_adapter_count": 0,
"model_can_approve_or_apply": False,
"operator_review_apply_tools_exposed": False,
},
"child_process": {
"pid": pid,
"readiness_verified": True,
"timed_out": False,
"exitcode": 0,
"termination": {"attempted": False},
"alive_after_readback": False,
"process_group_alive_after_readback": False,
"result_transport": "private_temp_file",
},
}
return {
"phase": phase,
"prompt": prompt,
"prompt_sha256": "prompt-hash",
"gateway": gateway,
"tool_invocations": invocations,
}
monkeypatch.setattr(lifecycle, "run_gateway_turn", fake_turn)
monkeypatch.setattr(
lifecycle,
"stage_lifecycle_proposal",
lambda _container, _database, _fixture: {
"created": False,
"idempotent_existing": True,
"marker_match_count": 1,
"proposal": _proposal(fixture, "pending_review"),
},
)
def fake_review(
_args: argparse.Namespace, _proposal_id: str
) -> tuple[subprocess.CompletedProcess[str], subprocess.CompletedProcess[str]]:
dry = subprocess.CompletedProcess([], 0, stdout="select kb_stage.approve_strict_proposal(...);", stderr="")
state["phase"] = "approved"
applied = subprocess.CompletedProcess([], 0, stdout="approved", stderr="")
return dry, applied
def fake_apply(
_args: argparse.Namespace, _proposal_id: str
) -> tuple[subprocess.CompletedProcess[str], subprocess.CompletedProcess[str]]:
dry = subprocess.CompletedProcess(
[],
0,
stdout=("select kb_stage.assert_approved_proposal(...); select kb_stage.finish_approved_proposal(...);"),
stderr="",
)
state["phase"] = "applied"
applied = subprocess.CompletedProcess([], 0, stdout="applied", stderr="")
return dry, applied
monkeypatch.setattr(lifecycle, "run_operator_review", fake_review)
monkeypatch.setattr(lifecycle, "run_guarded_apply", fake_apply)
def fake_counts(_container: str, _database: str, **_kwargs: Any) -> dict[str, int]:
if state["phase"] == "applied":
return dict(final_counts)
if state["phase"] == "approved":
return dict(approved_counts)
return dict(stage_counts)
monkeypatch.setattr(lifecycle.bound, "database_counts", fake_counts)
monkeypatch.setattr(
lifecycle.bound,
"read_tool_proof",
lambda _path, _container, _database, **_kwargs: {
"event_count": len(state["invocations"]) * 2,
"parse_errors": [],
"duplicate_invocation_ids": [],
"complete_start_end_pairing": True,
"invocations": list(state["invocations"]),
"invocation_count": len(state["invocations"]),
"all_bound_to_supplied_target": bool(state["invocations"]),
"database_read_only_required": False,
"all_completed_successfully": bool(state["invocations"]),
},
)
return state, {"runtime_root": runtime_root, "expected_rows": expected_rows}
def test_mocked_full_lifecycle_produces_private_passing_receipt_and_cleanup(
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
) -> None:
args = _args(tmp_path)
_state, proof = _install_fake_runtime(monkeypatch, tmp_path, args)
report = asyncio.run(lifecycle.run_lifecycle(args))
assert report["status"] == "pass", {
"errors": report["errors"],
"failed_checks": [name for name, passed in report["checks"].items() if not passed],
}
assert all(report["checks"].values())
assert report["final_canonical_rows"] == proof["expected_rows"]
assert report["target_deltas"]["baseline_to_final"] == {
"kb_stage.kb_proposals": 1,
lifecycle.APPROVAL_TABLE: 1,
"public.claim_edges": 0,
"public.claim_evidence": 1,
"public.claims": 1,
"public.reasoning_tools": 0,
"public.sources": 1,
}
assert report["production_invariants"]["guard_snapshot_unchanged"] is True
assert report["isolated_restart"]["same_persisted_session_id"] is True
assert report["cleanup"]["active_gateway_registry_clear"] is True
assert not os.path.lexists(proof["runtime_root"])
retained = json.loads(args.output.read_text(encoding="utf-8"))
assert retained["status"] == "pass"
assert args.output.stat().st_mode & 0o777 == 0o600
def test_mocked_lifecycle_fails_if_target_name_is_rebound_after_execution(
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
) -> None:
args = _args(tmp_path)
_install_fake_runtime(monkeypatch, tmp_path, args)
calls = 0
def replaced_identity(_container: str) -> dict[str, Any]:
nonlocal calls
calls += 1
return {
"id": "replacement-id",
"name": args.container,
"image": "postgres",
"labels": {
lifecycle.bound.DISPOSABLE_LABEL_KEY: lifecycle.bound.DISPOSABLE_LABEL_VALUE,
},
"network_mode": "none",
"ports": {},
"mount_sources": ["/replacement-data"],
"started_at": "replacement-start",
}
monkeypatch.setattr(lifecycle.bound, "container_identity", replaced_identity)
report = asyncio.run(lifecycle.run_lifecycle(args))
assert calls == 1
assert report["status"] == "fail"
assert any("pinned container identity" in item["message"] for item in report["errors"])
assert report["checks"]["target_container_name_still_resolves_to_pinned_identity"] is False
def test_mocked_lifecycle_fails_if_t1_changes_database_before_authorized_stage(
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
) -> None:
args = _args(tmp_path)
_install_fake_runtime(monkeypatch, tmp_path, args)
original_snapshot = lifecycle.bound.database_guard_snapshot
clone_reads = 0
def mutate_after_t1(container: str, database: str, **kwargs: Any) -> dict[str, Any]:
nonlocal clone_reads
snapshot = original_snapshot(container, database, **kwargs)
if container != lifecycle.bound.PRODUCTION_CONTAINER:
clone_reads += 1
if clone_reads == 2:
snapshot = json.loads(json.dumps(snapshot))
snapshot["rowset_md5"]["kb_stage.kb_proposals"] = "unexpected-t1-write"
return snapshot
monkeypatch.setattr(lifecycle.bound, "database_guard_snapshot", mutate_after_t1)
report = asyncio.run(lifecycle.run_lifecycle(args))
assert report["status"] == "fail"
assert any("T1 changed clone database state" in item["message"] for item in report["errors"])
assert report["checks"]["T1_database_unchanged_and_staging_absent"] is False
def test_source_contains_no_direct_canonical_write_service_restart_or_delivery_adapter() -> None:
source = (REPO_ROOT / "scripts" / "run_leo_clone_lifecycle_checkpoint.py").read_text(encoding="utf-8").lower()
assert "insert into public." not in source
assert "update public." not in source
assert "delete from public." not in source
assert "systemctl restart" not in source
assert "send_message(" not in source
assert "guarded._approve" in source
assert "guarded._apply" in source
def test_parse_args_requires_one_explicit_command_boundary(tmp_path: Path) -> None:
args = lifecycle.parse_args(
[
"--container",
"teleo-disposable-clone",
"--db",
"teleo",
"--output",
str(tmp_path / "receipt.json"),
"--copy-model-auth",
"--operator-review",
"--guarded-apply",
]
)
assert args.container == "teleo-disposable-clone"
assert args.db == "teleo"
assert args.copy_model_auth is True
assert args.operator_review is True
assert args.guarded_apply is True
assert lifecycle.MARKER_RE.fullmatch(args.run_marker)
assert lifecycle.MARKER_RE.fullmatch(args.conversation_marker)
assert args.run_marker != args.conversation_marker

View file

@ -0,0 +1,198 @@
from __future__ import annotations
import copy
import hashlib
import sys
from pathlib import Path
from typing import Any
import pytest
REPO_ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(REPO_ROOT / "scripts"))
import stage_normalized_proposal as stage # noqa: E402
PARENT_ID = "77777777-7777-4777-8777-777777777701"
AGENT_ID = "11111111-1111-4111-8111-111111111111"
DOCUMENT = (
"Operating note: a knowledge change becomes canonical only after reviewer approval and guarded apply. "
"Review alone does not change canonical rows."
)
POST = "The moment a reviewer approves a Leo proposal, it is already canonical. No apply step is needed."
def _parent() -> dict[str, Any]:
return {
"id": PARENT_ID,
"proposal_type": "attach_evidence",
"status": "pending_review",
"proposed_by_handle": "leo",
"proposed_by_agent_id": AGENT_ID,
"channel": "clone_composition",
"source_ref": "composition-fixture",
"payload": {
"claim_candidates": [
{
"claim_key": "guarded_apply_required",
"text": "Approval alone does not make a knowledge change canonical; guarded apply is required.",
"type": "structural",
"confidence": 0.98,
"evidence": [{"source_key": "operating_document", "role": "grounds"}],
"edges": [
{
"edge_type": "contradicts",
"target": "approval_is_canonical_post",
}
],
},
{
"claim_key": "approval_is_canonical_post",
"text": "A reviewer approval immediately makes a Leo proposal canonical.",
"type": "empirical",
"confidence": 0.4,
"evidence": [{"source_key": "contradictory_post", "role": "grounds"}],
},
],
"source_candidates": [
{
"source_key": "operating_document",
"source_type": "article",
"storage_path": "/composition/operating-note.txt",
"title": "Operating note",
"source_quality": "internal_operating_document",
"key_excerpt": DOCUMENT,
"content_sha256": hashlib.sha256(DOCUMENT.encode()).hexdigest(),
},
{
"source_key": "contradictory_post",
"source_type": "post",
"url": "https://x.example.test/composition/1",
"title": "Contradictory post",
"source_quality": "unverified_social_post",
"key_excerpt": POST,
"content_sha256": hashlib.sha256(POST.encode()).hexdigest(),
},
],
"dedupe_conflict_assessment": {
"duplicate_claim_keys": [],
"conflicts": [
{
"from_claim_key": "guarded_apply_required",
"to_claim_key": "approval_is_canonical_post",
"relationship": "contradicts",
}
],
},
},
}
def test_prepare_normalized_child_is_deterministic_hash_bound_and_applyable() -> None:
first = stage.prepare_normalized_child(_parent())
second = stage.prepare_normalized_child(copy.deepcopy(_parent()))
assert first == second
assert first["status"] == "pending_review"
assert first["proposal_type"] == "approve_claim"
assert first["source_ref"].startswith(f"normalized:{PARENT_ID}:")
payload = first["payload"]["apply_payload"]
assert len(payload["claims"]) == 2
assert len(payload["sources"]) == 4
assert len(payload["evidence"]) == 4
assert len(payload["edges"]) == 1
assert payload["edges"][0]["edge_type"] == "contradicts"
source_hashes = {row["hash"] for row in payload["sources"]}
assert hashlib.sha256(DOCUMENT.encode()).hexdigest() in source_hashes
assert hashlib.sha256(POST.encode()).hexdigest() in source_hashes
assert payload["normalization_manifest"]["parent_packet_sha256"] == stage.canonical_sha256(_parent())
def test_invalid_source_hash_refuses_to_prepare_a_child() -> None:
parent = _parent()
parent["payload"]["source_candidates"][0]["content_sha256"] = "bad"
with pytest.raises(stage.bound.CheckpointError, match="did not normalize"):
stage.prepare_normalized_child(parent)
def test_stage_sql_validates_exact_pending_row_before_commit_and_never_writes_public() -> None:
child = stage.prepare_normalized_child(_parent())
sql = stage.build_stage_sql(child)
lowered = sql.lower()
assert "insert into kb_stage.kb_proposals" in lowered
assert "on conflict (id) do nothing" in lowered
assert "normalized stage validation failed" in lowered
assert lowered.index("raise exception") < lowered.index("commit;")
assert child["id"] in sql
assert child["payload_sha256"] in sql
assert "insert into public." not in lowered
assert "update public." not in lowered
assert "delete from public." not in lowered
def test_stage_normalized_child_binds_to_full_disposable_container_id(
monkeypatch: pytest.MonkeyPatch,
) -> None:
target = {
"id": "clone-container-id",
"name": "composition-clone",
"labels": {stage.bound.DISPOSABLE_LABEL_KEY: stage.bound.DISPOSABLE_LABEL_VALUE},
"network_mode": "none",
"ports": {},
"mount_sources": ["/clone-data"],
}
production = {
"id": "production-container-id",
"name": stage.bound.PRODUCTION_CONTAINER,
"labels": {},
"network_mode": "bridge",
"ports": {},
"mount_sources": ["/production-data"],
}
monkeypatch.setattr(
stage.bound,
"container_identity",
lambda name: target if name == "composition-clone" else production,
)
observed: dict[str, Any] = {}
def fake_psql(container: str, database: str, sql: str) -> dict[str, Any]:
child = stage.prepare_normalized_child(_parent())
observed.update({"container": container, "database": database, "sql": sql})
return {
"created": True,
"proposal_id": child["id"],
"proposal_type": child["proposal_type"],
"status": "pending_review",
"source_ref": child["source_ref"],
"payload_sha256": child["payload_sha256"],
}
monkeypatch.setattr(stage.bound, "_psql_json", fake_psql)
result = stage.stage_normalized_child("composition-clone", "teleo", _parent())
assert observed["container"] == "clone-container-id"
assert observed["database"] == "teleo"
assert result["bound_container_id"] == "clone-container-id"
assert result["database_receipt"]["created"] is True
def test_main_rejects_the_production_container_without_reading_the_packet(tmp_path: Path) -> None:
missing = tmp_path / "missing.json"
assert (
stage.main(
[
"--container",
stage.bound.PRODUCTION_CONTAINER,
"--db",
stage.bound.PRODUCTION_DB,
"--proposal-file",
str(missing),
]
)
== 2
)