For AI agents: the complete documentation index is available at /llms.txt, the full documentation bundle is available at /llms-full.txt, and this page is available as Markdown at /reference/docs/decisions/0040-platform-layout.md.

0040 — Platform layout: Ansible for the cluster, Argo CD for services

Status: Proposed. The design was agreed in review on 13 October 2026 and is implemented in stages. Date: 13 October 2026.

Context

The dev-box platform grew one stage at a time: K3s, Rook/Ceph, the host edge, Istio and Argo CD (0037, 0038, 0039). GitLab, Waldur, the retained Themelio code, Harbor and the inference services come next. Without a rule for where they belong, each would add its own Make target, guard or marker, and the deployment order would stay spread over code, comments and host files.

The legacy deployment separated Ansible (devops/ansible: inventory, playbooks, roles) from GitOps (gitops/apps, gitops/manifests, one root Application per environment). That layout is familiar to the team and fits staging and production.

Decision

  1. Ansible installs only the cluster layers: Kubernetes, storage, Istio, the host edge and Argo CD, plus GPU setup when the inventory has GPU nodes. Each layer is one role and one playbook in devops/ansible, named after the layer. GPU-host roles must be selected through an explicit GPU group. They never run against CPU-only development VMs. Host drivers, kernel modules, the NVIDIA Container Toolkit and containerd runtime configuration remain host-owned.
  2. Argo CD installs every service. Each service is a self-contained Application with its own manifests and per-environment values. Sync waves order services; optional services are Kustomize components selected per environment. The cluster-side NVIDIA GPU Operator is an opt-in infrastructure Application: its CRDs, controller, ClusterPolicy and operands have an exact GitOps revision, while host GPU software stays outside it. Adoption is suspended until the exact live release and policy diff are reviewed; enabling it is not implicit in an ordinary delivery.
  3. Optional host laboratories are separate. A laboratory keeps its Ansible beside its code (dx/<service>/<laboratory>/ansible) and has its own command, run after make deploy. It checks that the platform is present, is idempotent, and is never run or removed by make deploy. A laboratory may use platform roles; the platform never depends on a laboratory.
  4. The layout follows the legacy one (devops/ansible, gitops/<env>-root-app.yaml, gitops/apps/{base,components,overlays}, gitops/manifests/<service>), filled with this platform's own roles. legacy/ remains a read-only reference.
  5. The repository is the single source of truth, except host addresses and secrets, which stay in the private inventory and secret store as before.

Stages

Each stage is its own pull request, in this order:

  1. Move the platform Ansible to devops/ansible with layer-named roles and the Icarus laboratory's Ansible to dx/metallo/icarus-runtime/ansible; remove the skeleton's unused K3s role; enforce the one-way laboratory rule.
  2. Replace the Helm root chart with per-environment Kustomize roots and move the GitOps tree into the legacy layout: gitops/devbox-root-app.yaml, gitops/apps/{base,overlays/devbox} and gitops/manifests/{argocd,routes}. This changes the reviewed Argo contracts in 0039, so it has its own live qualification.
  3. GitLab, Waldur and their databases as Argo CD services, logging in through the shared AAI and exposed through the existing catalogue.
  4. The retained Themelio as an optional Argo CD service using them.
  5. Complete the layout without changing what is deployed: the new platform's manifests move to gitops/manifests/pharos/{base,overlays/{devbox,k3s,local}}, the route catalogue and its environment values to gitops/manifests/routes/{base,overlays/devbox}, and the Istio pins to the istio role. Only gitops/<env>-root-app.yaml, gitops/apps and gitops/manifests remain at the GitOps root.
  6. The new platform becomes the Argo CD service pharos in the pharos namespace, replacing pharos-fresh and metallo-devbox, with make reconcile to deliver a commit and build only the images it changes.
  7. Routes and policies move from the catalogue to each service; the bastion forwards, the Istio gateway terminates TLS, and management UIs use a mgmt. host.
  8. The AAI becomes an Argo CD service, preserving its issuer, CA and data.
  9. Personal workstation and dev-box controllers, local AAI and shared tools live in dx/common. Service-specific qualifications and laboratories live in dx/<service>. Shared deployment libraries live in devops/platform; Ansible roles and canonical GitOps manifests retain their deployment ownership.

Consequences

Positive: one place per kind of change; new services touch only GitOps; the layout carries over to staging and production; laboratories cannot become hidden platform dependencies.

Negative: paths and role names change for current dev-box users and open branches; each stage needs rebasing and, from stage 3, live requalification.

Neutral: host paths, markers, installed resources and dev-box commands keep their names in stage 1.

Page source

Source snapshot · Revision f928834c8.