# 0037 — Dev-box storage from the final deployment's Rook/Ceph role

Status: Accepted for development use. Date: 11 October 2026.

Dev-box providers need block volumes and S3 buckets. Without a shared substrate,
each one brings its own: an omnibus GitLab keeping LFS on its volume, Garage or
MinIO for S3, and `local-path` volumes for databases. Code written against those
local providers then differs from the final deployment, which uses Rook/Ceph.

Each dev-box VM has a second, empty disk for this purpose. The `storage`
role installs Rook/Ceph on it and provides `ceph-block` volumes, `ceph-filesystem`
volumes and `ceph-bucket` object buckets through Ceph RGW. The role is a copy of
the final deployment's `storage` role from `legacy/devops/ansible/roles/storage`,
cluster-wide flow only, with two later changes from the GB10 cluster: a 16 MiB
CephFS readahead and a 16Gi OSD memory limit. The Minikube flow is not copied; the
legacy local setup never installed Ceph.

Only inventory values differ from a multi-node installation: one node is both
storage and monitor node, pools keep one replica, and the explicitly named disk
is the only OSD. The role adds variables whose defaults keep the original
behaviour: single-replica safety, Helm value overrides, verified chart archives
and an optional dashboard route. The dev box pins the Ceph and operator images
by digest and disables Prometheus integration, which it does not run. As in the
final deployment, `ceph-block` is the default class: K3s no longer restores
`local-path` as a second default, and claims made before storage keep their
`local-path` volumes until they are recreated. It points the
CSI node plugins at the dev box's K3s kubelet directory and silences only the
single-replica health warnings, so a healthy dev box reports `HEALTH_OK`. Its OSD
keeps the chart's 4Gi memory limit: Rook derives the OSD's cache target from the
limit, a 32 GB VM also runs GitLab, Waldur and Keycloak, and measured CephFS reads
were no faster with 16Gi. The readahead alone made them about 1.5 to 2 times faster. The role
waits for the pool, filesystem and object store to be ready before reporting
success. Rook leaves a bucket claim's `BUCKET_REGION` empty; S3 clients sign with
the `ceph-bucket` class's region.

The dev box's disks are themselves Ceph volumes of the hosting OpenStack cloud,
so each in-VM monitor and OSD adds synchronous writes to shared storage.

Kubernetes tasks run in the Metallo controller through the target tunnel; the VM
needs neither Helm nor Kubernetes Python libraries. Before Rook starts, a guard on
the VM accepts only a whole, unmounted disk without partitions that is empty or
already a Ceph OSD, and never the root disk. The first successful run records the
disk for the target; moving Ceph to another disk needs an explicit migration.

First setup runs `make deploy` from the repository root. Its ordered stages install
K3s, then storage, then check block, filesystem and S3 storage, and each is safe to
repeat. It runs before application setup, so the stack's claims use Ceph without
naming a class. Later platform layers join the same command as further stages. New dev-box providers use the default class, `ceph-filesystem` or
`ceph-bucket` rather than adding local storage providers. Existing laboratories and parallel-operation
profiles move to it in their own changes.

## Optional storage sources

Dedicated-disk storage remains the default. An explicit `STORAGE_MODE=file`
preallocates a target-owned file on the internal filesystem and attaches it through
a persistent loop-device binding. Its size and identity are recorded before use;
setup refuses unrelated files, partial allocation and changes to the bound source.
This single-node development option has the same single-replica limitation.

`STORAGE_MODE=existing` validates and binds a healthy Rook/Ceph provider with the
three named StorageClasses and the dashboard required by the shared routes.
It preserves that installation and the host's default StorageClass. New platform
claims explicitly select Ceph. Provider extension and foreign namespace handover
are separate operator actions; setup never adopts them implicitly. The personal
[ARM64 profile](../../dx/arm64-singledrive/README.md) records its qualification.