teleo/teleo-infrastructure
8
0
Fork 0
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions 1

Merge pull request #209 from living-ip/codex/review-gcp-pr206-cache-isolation-20260721

Browse source 
Retain inspected digest references in shared Docker caches
This commit is contained in:
Fawaz 2026-07-21 00:14:43 -07:00 committed by GitHub
commit 2abe48b7e9
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 120 additions and 127 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

29
docs/gcp-leoclean-nosend-service-package.md
View file

@ -186,13 +186,15 @@ The Docker binary, config directory, and local `unix://` daemon endpoint are
explicit inputs; inherited `DOCKER_*` overrides are removed. The pull does not
start a container, install a unit, restart a service, or contact Cloud SQL.
Registry authentication and this pull require a separate live-read approval.
The exact repository-digest reference must be absent before inspection; use a
fresh isolated Docker context if it is already cached. While holding an
advisory lock on the explicit Docker config directory, the command pulls,
records the dynamically observed config ID, then removes the reference only if
that ID is still unchanged and proves it absent before publishing the bundle.
Content-addressable layers may remain. A failed pull, inspection, or cleanup
returns a bounded error without forwarding Docker stderr.
While holding an advisory lock on the explicit Docker config directory, the
command pulls and records the dynamically observed config ID. It intentionally
retains the exact repository-digest reference: Docker's image cache belongs to
the daemon, so a config-directory lock cannot prove that another client did not
start relying on the same immutable reference after the precheck. Repeated runs
re-pull and re-inspect the same digest without deleting shared cache state.
Content-addressable layers and the verified digest reference may remain. A
failed pull or inspection returns a bounded error without forwarding Docker
stderr.
Tags, alternate repositories, alternate projects/instances/service accounts,
extra descriptor fields, wrong-architecture images, missing repository-digest
@ -202,7 +204,8 @@ values are rejected.
## Offline evidence and live gate
Offline tests prove descriptor and inspection strictness, explicit local Docker
authority, cache-reference rollback, atomic bundle publication, artifact and synthetic identity binding,
authority, non-destructive cache reuse, atomic bundle publication, artifact and
synthetic identity binding,
digest-only images, pinned runtime versions, a secret-free context, the
capability-drop contract, no published ports, no Docker-socket mount, no
environment file, and no production/VPS/Telegram target. CI also builds exactly
@ -231,7 +234,7 @@ virtual environment.
Publishing the draft branch and stacked PR is only a review handoff. The next
deployment-evidence step must execute one authorized remote registry read plus
the reversible local exact-reference cache mutation against the staging digest.
the retained local exact-reference cache mutation against the staging digest.
A later installer slice must prove installation and rollback consume the same
receipt. Live installation, Artifact Registry publication, service restart,
IAM/secret changes, Cloud SQL access, and proposal staging remain separate
@ -240,10 +243,10 @@ approvals.
## Rollback and sunset
This slice's rollback boundary is removal of the unpublished atomic output
bundle plus removal of the exact repository-digest reference that this command
introduced into an initially clean, advisory-locked cache; content-addressable
layers may remain. It installs no unit
and changes no running service. Restoring a prior descriptor, unit, and digest
bundle. The verified repository-digest reference and content-addressable layers
may remain in the local daemon cache so the finalizer never deletes shared cache
state it cannot exclusively own. It installs no unit and changes no running
service. Restoring a prior descriptor, unit, and digest
belongs to the later installer slice, which must never restart or edit the
production Leo, Telegram, or VPS services.

64
ops/gcp_leoclean_nosend_package.py
View file

@ -1741,36 +1741,6 @@ def _local_image_reference_config_id(
return config_id
def _remove_new_image_reference(
docker_binary: Path,
docker_host: str,
docker_config: Path,
image_reference: str,
expected_config_id: str,
) -> None:
observed_config_id = _local_image_reference_config_id(
docker_binary,
docker_host,
docker_config,
image_reference,
)
if observed_config_id is None:
return
_require(observed_config_id == expected_config_id, "new registry image reference changed before cleanup")
_run_docker(
docker_binary,
docker_host,
docker_config,
["image", "rm", image_reference],
check=True,
timeout=120,
)
_require(
_local_image_reference_config_id(docker_binary, docker_host, docker_config, image_reference) is None,
"new registry image reference cleanup did not pass",
)
def inspect_registry_image(
image_input: dict[str, Any],
image_reference: str,
@ -1806,17 +1776,8 @@ def inspect_registry_image(
"image input differs from the expected reviewed input SHA-256",
)
_validate_docker_authority(docker_binary, docker_host, docker_config)
prestate_observed = False
pulled_config_id: str | None = None
inspection: dict[str, Any] | None = None
primary_error: BaseException | None = None
with _DockerCacheLock(docker_config):
try:
_require(
_local_image_reference_config_id(docker_binary, docker_host, docker_config, image_reference) is None,
"exact registry image reference must be absent before inspection",
)
prestate_observed = True
_run_docker(
docker_binary,
docker_host,
@ -1849,30 +1810,7 @@ def inspect_registry_image(
"inspected image differs from the expected reviewed config digest",
)
except (OSError, subprocess.SubprocessError, PackageError) as exc:
primary_error = exc
finally:
try:
if prestate_observed:
if pulled_config_id is None:
pulled_config_id = _local_image_reference_config_id(
docker_binary,
docker_host,
docker_config,
image_reference,
)
if pulled_config_id is not None:
_remove_new_image_reference(
docker_binary,
docker_host,
docker_config,
image_reference,
pulled_config_id,
)
except (OSError, subprocess.SubprocessError, PackageError) as cleanup_error:
raise PackageError("immutable registry image cleanup failed") from cleanup_error
if primary_error is not None:
raise PackageError("immutable registry image inspection failed") from primary_error
_require(inspection is not None, "immutable registry image inspection failed")
raise PackageError("immutable registry image inspection failed") from exc
return inspection

154
tests/test_gcp_leoclean_nosend_package.py
View file

@ -1064,7 +1064,7 @@ def test_local_image_inventory_rejects_inventory_and_inspection_disagreement(
@pytest.mark.parametrize("config_digest", ["sha256:" + "3" * 64, "sha256:" + "a" * 64])
def test_registry_inspection_uses_explicit_local_authority_and_removes_new_reference(
def test_registry_inspection_absent_pull_dependent_retains_new_reference(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
config_digest: str,
@ -1075,7 +1075,7 @@ def test_registry_inspection_uses_explicit_local_authority_and_removes_new_refer
prefix = [str(docker_binary), "--host", docker_host, "--config", str(docker_config)]
calls: list[list[str]] = []
environments: list[dict[str, str]] = []
state = {"present": False}
state = {"present": False, "dependent": False, "inspection_calls": 0}
payload = json.dumps(
[docker_inspection_fixture(image_input, reference, config_digest=config_digest)]
).encode()
@ -1099,6 +1099,9 @@ def test_registry_inspection_uses_explicit_local_authority_and_removes_new_refer
stderr=b"",
)
if operation[:2] == ["image", "inspect"]:
state["inspection_calls"] += 1
if state["inspection_calls"] == 2:
state["dependent"] = True
return subprocess.CompletedProcess(
arguments,
0,
@ -1109,9 +1112,7 @@ def test_registry_inspection_uses_explicit_local_authority_and_removes_new_refer
state["present"] = True
return subprocess.CompletedProcess(arguments, 0, stdout=b"pulled\n", stderr=b"")
if operation[:2] == ["image", "rm"]:
assert "--force" not in operation
state["present"] = False
return subprocess.CompletedProcess(arguments, 0, stdout=b"removed\n", stderr=b"")
raise AssertionError("digest reference must never be removed")
raise AssertionError(operation)
monkeypatch.setattr(package.subprocess, "run", fake_run)
@ -1128,15 +1129,60 @@ def test_registry_inspection_uses_explicit_local_authority_and_removes_new_refer
)
assert inspection["status"] == "pass"
assert state["present"] is False
assert state == {"present": True, "dependent": True, "inspection_calls": 2}
assert [call[5:] for call in calls if call[5] == "pull"] == [
["pull", "--quiet", "--platform", package.PLATFORM, reference]
]
assert all(not any(name.startswith("DOCKER_") for name in environment) for environment in environments)
assert [call[5:] for call in calls if call[5:7] == ["image", "rm"]] == [["image", "rm", reference]]
assert not any(call[5:7] == ["image", "rm"] for call in calls)
def test_registry_inspection_rejects_reviewed_config_digest_mismatch_and_cleans_reference(
def test_registry_inspection_retains_preexisting_reference(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
image_input = image_input_fixture(tmp_path)
reference = f"{package.IMAGE_REPOSITORY}@sha256:{IMAGE_DIGEST}"
docker_binary, docker_host, docker_config = docker_authority(tmp_path)
payload = json.dumps([docker_inspection_fixture(image_input, reference)]).encode()
state = {"present": True, "pulls": 0}
def fake_run(arguments: list[str], **_kwargs):
operation = arguments[5:]
if operation[:2] == ["image", "ls"]:
return subprocess.CompletedProcess(
arguments,
0,
stdout=docker_inventory_fixture(reference, present=state["present"]),
stderr=b"",
)
if operation[:2] == ["image", "inspect"]:
return subprocess.CompletedProcess(arguments, 0, stdout=payload, stderr=b"")
if operation[0] == "pull":
state["pulls"] += 1
return subprocess.CompletedProcess(arguments, 0, stdout=b"pulled\n", stderr=b"")
if operation[:2] == ["image", "rm"]:
raise AssertionError("digest reference must never be removed")
raise AssertionError(operation)
monkeypatch.setattr(package.subprocess, "run", fake_run)
inspection = package.inspect_registry_image(
image_input,
reference,
expected_revision=image_input["runtime"]["teleo_git_head"],
expected_input_sha256=image_input["input_sha256"],
expected_config_digest="sha256:" + "3" * 64,
docker_binary=docker_binary,
docker_host=docker_host,
docker_config=docker_config,
)
assert inspection["status"] == "pass"
assert state == {"present": True, "pulls": 1}
def test_registry_inspection_rejects_config_digest_mismatch_and_retains_reference(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
@ -1180,10 +1226,10 @@ def test_registry_inspection_rejects_reviewed_config_digest_mismatch_and_cleans_
)
assert "expected reviewed config digest" in str(raised.value.__cause__)
assert state == {"present": False, "removed": True}
assert state == {"present": True, "removed": False}
def test_registry_inspection_failure_is_redacted_and_cleans_new_reference(
def test_registry_inspection_pull_failure_is_redacted_and_does_not_remove_reference(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
@ -1226,11 +1272,11 @@ def test_registry_inspection_failure_is_redacted_and_cleans_new_reference(
docker_config=docker_config,
)
assert state["present"] is False
assert state["present"] is True
assert "credential=value" not in str(raised.value)
def test_registry_inspection_validation_failure_removes_new_reference(
def test_registry_inspection_validation_failure_retains_reference(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
@ -1280,10 +1326,10 @@ def test_registry_inspection_validation_failure_removes_new_reference(
docker_config=docker_config,
)
assert state == {"present": False, "removed": True}
assert state == {"present": True, "removed": False}
def test_registry_inspection_cleanup_failure_prevents_success(
def test_registry_inspection_never_calls_image_rm_after_success(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
@ -1318,20 +1364,22 @@ def test_registry_inspection_cleanup_failure_prevents_success(
monkeypatch.setattr(package.subprocess, "run", fake_run)
with pytest.raises(package.PackageError, match="cleanup failed"):
package.inspect_registry_image(
image_input,
reference,
expected_revision=image_input["runtime"]["teleo_git_head"],
expected_input_sha256=image_input["input_sha256"],
expected_config_digest="sha256:" + "3" * 64,
docker_binary=docker_binary,
docker_host=docker_host,
docker_config=docker_config,
)
inspection = package.inspect_registry_image(
image_input,
reference,
expected_revision=image_input["runtime"]["teleo_git_head"],
expected_input_sha256=image_input["input_sha256"],
expected_config_digest="sha256:" + "3" * 64,
docker_binary=docker_binary,
docker_host=docker_host,
docker_config=docker_config,
)
assert inspection["status"] == "pass"
assert state["present"] is True
def test_registry_inspection_refuses_to_remove_changed_reference(
def test_registry_inspection_has_no_post_inspection_cleanup_window(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
@ -1373,23 +1421,24 @@ def test_registry_inspection_refuses_to_remove_changed_reference(
monkeypatch.setattr(package.subprocess, "run", fake_run)
with pytest.raises(package.PackageError, match="cleanup failed"):
package.inspect_registry_image(
image_input,
reference,
expected_revision=image_input["runtime"]["teleo_git_head"],
expected_input_sha256=image_input["input_sha256"],
expected_config_digest="sha256:" + "3" * 64,
docker_binary=docker_binary,
docker_host=docker_host,
docker_config=docker_config,
)
inspection = package.inspect_registry_image(
image_input,
reference,
expected_revision=image_input["runtime"]["teleo_git_head"],
expected_input_sha256=image_input["input_sha256"],
expected_config_digest="sha256:" + "3" * 64,
docker_binary=docker_binary,
docker_host=docker_host,
docker_config=docker_config,
)
assert inspection["status"] == "pass"
assert state["present"] is True
assert state["removed"] is False
assert state["inventory_calls"] == 1
def test_registry_inspection_rejects_preexisting_reference_without_mutation(
def test_registry_inspection_accepts_preexisting_reference_without_removal(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
@ -1411,24 +1460,27 @@ def test_registry_inspection_rejects_preexisting_reference_without_mutation(
)
if operation[:2] == ["image", "inspect"]:
return subprocess.CompletedProcess(arguments, 0, stdout=payload, stderr=b"")
if operation[0] == "pull":
return subprocess.CompletedProcess(arguments, 0, stdout=b"pulled\n", stderr=b"")
if operation[:2] == ["image", "rm"]:
raise AssertionError("digest reference must never be removed")
raise AssertionError(operation)
monkeypatch.setattr(package.subprocess, "run", fake_run)
with pytest.raises(package.PackageError, match="inspection failed") as raised:
package.inspect_registry_image(
image_input,
reference,
expected_revision=image_input["runtime"]["teleo_git_head"],
expected_input_sha256=image_input["input_sha256"],
expected_config_digest="sha256:" + "3" * 64,
docker_binary=docker_binary,
docker_host=docker_host,
docker_config=docker_config,
)
inspection = package.inspect_registry_image(
image_input,
reference,
expected_revision=image_input["runtime"]["teleo_git_head"],
expected_input_sha256=image_input["input_sha256"],
expected_config_digest="sha256:" + "3" * 64,
docker_binary=docker_binary,
docker_host=docker_host,
docker_config=docker_config,
)
assert "must be absent" in str(raised.value.__cause__)
assert not any(operation and operation[0] == "pull" for operation in operations)
assert inspection["status"] == "pass"
assert any(operation and operation[0] == "pull" for operation in operations)
assert not any(operation[:2] == ["image", "rm"] for operation in operations)