> For AI agents: the complete documentation index is available at /llms.txt, the full documentation bundle is available at /llms-full.txt. # DD-0045 — Development loop interface: image catalogue, deployment receipt and takeover lease **Related ADR:** [ADR-0045](/reference/docs/decisions/0045-development-loop.md). **Status:** Draft for review. The lease, mode, expiry and catalogue decisions were reviewed with danias before implementation; dev-box qualification follows each pull request. **Date:** 2026-10-14. ## Scope Three commands share one dev box: | Command | Input | Result | | ---------------- | --------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- | | `make deploy` | The committed checkout, the target's profiles | Cluster layers (Ansible), AAI, provider tokens, then the same delivery as `make reconcile` | | `make reconcile` | A commit | The commit delivered to the in-cluster Git server, the images it changes built from that commit, Argo CD synced to it | | takeover | The local checkout with uncommitted edits | Selected workloads run on the workstation (native or Docker runtime) against the dev box, under a lease | On macOS, `deploy` and `reconcile` send the exact Git commit as a bundle to an SSH Linux runner. The runner uses the existing Linux tools and the same receipt, image and lease protocols. Its owned checkout is separate from operator source; an explicit `REMOTE_ROOT` can reuse an existing installation's private state. `make reconcile` and takeover share the same delivery interfaces. They agree on three interfaces, described here: the image catalogue, the deployment receipt and the takeover lease. The workstation watcher follows saved files directly. The root `make tilt` adds a dashboard: saved files invoke takeover, while hand-back, delivery, status and log snapshots remain explicit actions. ## Image catalogue One file, `gitops/manifests/images.yaml`, lists every image the platform builds. It replaces `devops/parallel/build-inventory.json` from #626 and the inputs that `dx/common/scripts/devbox/source.py` and `themelio_compat.py` encode today. It sits in `gitops/manifests`, since only root Applications, `apps` and `manifests` may appear at the GitOps root. ```yaml platform: linux/amd64 images: api: image: pharos/api # as the manifests name it application: pharos context: . dockerfile: services/api/Dockerfile inputs: [services/api, contracts, .dockerignore] realtime-gateway: image: pharos/realtime-gateway application: pharos context: . dockerfile: services/api/Dockerfile args: {APP: realtime-gateway} inputs: [services/api, contracts, .dockerignore] compat-web: image: docker.io/pharos/themelio-compat-web application: themelio-compat context: compat/themelio dockerfile: compat/themelio/Dockerfile target: web inputs: [compat/themelio] # the Dockerfile copies its whole context # ... themelio, beto, metallo, web, compat-gitlab-root-actor, compat-k8s-root-actor ``` - **`inputs`** are the paths whose content decides the image, including shared modules, contracts and lockfiles. The Dockerfile, `args`, `target` and `platform` are always inputs. Test files and documentation are excluded by the existing `runtime_path` rule. - **The fingerprint** of an image is the SHA-256 of its inputs' paths, modes and contents, plus the Dockerfile, `args`, `target` and `platform`. It is computed from a Git tree for `make reconcile` and from the working tree for takeover, with the same function. - **A check** (`check_platform_gitops.py`) fails when a Dockerfile `COPY`s a path that is not an input, when an input does not exist, or when an image named by `gitops/manifests` is not in the catalogue. - **`application`** names the Argo CD Application whose workloads use the image; the lease and the receipt use it. ## Deployment receipt `make reconcile` and `make deploy` write one receipt per delivery, and takeover reads the last one as its baseline. - **In the cluster:** the existing `pharos-platform-ownership` ConfigMap gains the delivered commit (already its `revision`) and the image map. That map is part of the root's inputs, so the root, not a patch on a child, passes each digest to its Application's `kustomize.images`, as #679 does for `themelio-compat`. - **Locally:** `.local/metallo//receipt.json` repeats it with each image's fingerprint, so takeover can compare without a cluster round trip: ```json {"schema": 1, "commit": "<40 hex>", "delivered": "2026-10-14T12:00:00Z", "images": {"compat-web": {"fingerprint": "<64 hex>", "reference": "registry.pharos-build.svc.cluster.local:5000/compat-web@sha256:<64 hex>"}}} ``` - **Order:** the receipt and the root's inputs change only after every changed image has built and been pushed. A failed build leaves the previous delivery in place. - **Takeover's comparison:** an image whose working-tree fingerprint equals the receipt's runs in the cluster. One that differs is a candidate for takeover if it supports a runtime there, or needs `make reconcile` if not (actors, migrations). ## Builds in the cluster - A BuildKit Job per changed image in the `pharos-build` namespace. Its context is the delivered commit, fetched from the in-cluster Git server, so the build uses the commit that was delivered and nothing is uploaded from the workstation. - An in-cluster registry in the same namespace, on a Ceph block volume, which also holds the BuildKit cache. k3s pulls from it through a `registries.yaml` mirror the `k3s` role installs, over the cluster network only. - Requests and limits are set per Job; builds run one at a time by default. - Images are named by digest in the root's inputs and never by tag. ## The `pharos` service `gitops/apps/base/pharos.yaml` is installed on every dev box on wave 3, in the bootstrap's `pharos` namespace; `apps/overlays/devbox` sets its path to `gitops/manifests/pharos/overlays/devbox`. The overlay removes the base's Namespace and the Minikube Ingress, so the web has no edge route until the new Themelio replaces `themelio-compat`. It adds the namespace's administration, in sync waves: | Wave | Resources | | ---- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | −4 | The credentials Job's account and Role (Secrets `get`, `create`, `patch`); the namespace-wide `dns` and `default-deny` policies; the Jobs' own egress and PostgreSQL's ingress from the provision Job | | −3 | Sync hook `pharos-credentials` (`scripts/cluster_credentials.py`) | | −2 | PostgreSQL's StatefulSet, Service and ServiceAccount | | −1 | Sync hook `pharos-database-provision` (`scripts/provision_contexts.py --secrets`) | | 0 | The workloads | - **Credentials.** Every key in `devops/secret-keys.json` has one value, kept in `pharos-generated`, and each `-credentials` Secret receives only its own keys, so shared keys agree. A key the inventory gains is added; an existing value is never changed, and a Secret that differs from the record is refused. A lost record is recovered from the workloads' Secrets. The Job prints names only. - **Databases.** The provision Job waits for PostgreSQL, then creates each context's role and database and applies its migrations from the bindings in `metallo-credentials`. It alone receives the administrator password. - **`database-admin` image.** PostgreSQL's client with Python, the inventory, the context registry, the migrations and the scripts, running as UID 70. `check_platform_gitops.py` requires every registered context's binding in the Job and the inventory, its migrations in the image's inputs, and every namespace-wide policy before the hooks. - **Transition.** `overlays/metallo-devbox` installs the same base in `metallo-devbox` for the Metallo controller's own loop, until its takeover moves to `pharos`. ## Takeover lease A takeover borrows the Applications whose workloads it replaces. While it holds them, Argo CD syncs neither those Applications automatically nor by hand. Every other Application, and the shared GitLab, Waldur, AAI and edge, keep syncing. **Mode: one developer per dev box.** A target has at most one lease. It covers every Application the takeover touches (`pharos`, `themelio-compat` or both). A second checkout is refused rather than given its own lease. ### The lease is a sync window The lease has two parts: - **A record** in the ownership ConfigMap: the holder (checkout and workstation), the Applications, the receipt's commit, when it was taken and when it expires. - **A deny sync window** on the Applications' AppProject, naming only the leased Applications, with `manualSync: false`. A deny window stops automated syncs, self-heal included, and with `manualSync: false` also manual ones, for those Applications only. The Applications' own sync policy does not change. The root therefore has nothing to restore while the lease is held, and Argo CD shows the leased Applications as OutOfSync without reverting them. The AppProjects belong to the Ansible bootstrap and the project-writer admission fence. The controller uses the existing bootstrap administrator identity, which the fence already permits; Argo CD service accounts remain denied. It preserves other windows and adds or removes only its own deny window with `manualSync: false`, matching the Applications in the ownership record. ### Expiry - A window has a start (`schedule`, a cron expression for the minute it begins) and a `duration`. The lease's window starts when it is acquired and lasts the lease's time to live, 12 hours by default. - While the takeover runs, it renews the lease every 5 minutes, moving the record's expiry and the window's `duration` forward. - If the workstation stops renewing (closed laptop, lost connection, crash), the window ends on its own. Argo CD then syncs the Applications again, and self-heal restores their workloads from the receipt's commit without any command. - The next command for the target (takeover, `make reconcile` or `make deploy`) removes the expired record and window and continues. A later takeover from the same checkout starts a new lease. ### Acquire 1. Refuse if an unexpired lease, an unfinished `make deploy` or `make reconcile`, or a different checkout already owns the target. 2. Write the record, then add the window. 3. Wait for any operation in progress on the leased Applications to finish, or end it if it is for a superseded revision, as the health wait does today. 4. Only then replace the workloads. ### While held `make reconcile` and `make deploy` refuse to deliver while an unexpired lease exists. The holder hands back first. ### Hand-back 1. Stop the local runtime. 2. Detach replacements and restore container aliases. Queue agent removal until every local runtime has stopped; namespace-scoped uninstall can then reconnect without disrupting another service. Journal each cleanup before proceeding. 3. Remove the window and then the record. Argo CD syncs the Applications to the receipt's commit, and self-heal restores their workloads. 4. Wait until the Applications are Synced and Healthy at that commit. ### Interruption and recovery - The record and a local journal mark each step, so a retried acquire or hand-back continues from the last completed step. - A window without a record, or a record without a window, is completed or removed by the next command for the target. - Qualification includes an interrupted acquire, an interrupted hand-back, a dropped connection, a sleeping workstation and an expired lease. ## Decisions from review - Manual sync is blocked only for the Applications under takeover, through the deny sync window; Argo CD's RBAC does not change. - One developer per dev box, so one lease per target. - A lease expires when its holder stops renewing it. - The catalogue's inputs as listed above.
Page source Source snapshot · Revision `f928834c8`.