0048 — Shared infrastructure on personal development boxes
Status: accepted
Context
The personal dev-box stack installed five PostgreSQL servers, two RabbitMQ servers and five Redis/Valkey servers. GitLab and Waldur also used their database bootstrap identities in application containers. Developers need a smaller stack with clear consumer boundaries and a repeatable clean installation.
This decision concerns the development environment. Official testing, staging and
production keep their environment-specific deployment policy under devops.
Decision
Argo CD installs pharos-infra, then pharos-aai, then their consumers, in the
root's sync waves. Delivery readiness follows decision 0039; see
Argo-started operations. Bootstrap-owned RBAC grants
the infrastructure credential Job access to the explicitly named consumer
namespaces. Application pods have no such authority.
One PostgreSQL 18.6 server supplies these databases:
Each consumer receives a distinct generated password. Public database connection rights are revoked. Runtime roles cannot create databases or roles, bypass row security or become superusers. Schema migration privileges remain with each consumer's existing migration mechanism. PostgreSQL administrator credentials belong to provider and administration jobs. Namespace-local Service aliases keep application connection names stable.
One RabbitMQ 4.1.4 server supplies the Pharos reference virtual host and Waldur's
/ Celery virtual host. Waldur 8.1.2 additionally creates UUID-named event virtual
hosts and users at runtime. Its account has no management or administrator tags.
A small infrastructure adaptor authenticates that account and permits only the
reviewed Waldur lifecycle operations. It filters listings and denies access to
Pharos resources, elevated user tags, global policies, plugins and shovels.
The adaptor grants the Waldur producer access when creating an event virtual host.
The broker administrator password remains inside infrastructure administration.
NetworkPolicy prevents Waldur from reaching the broker's direct management port;
its management Service points to the adaptor instead.
One persistent Valkey 8.1.3 server supplies Pharos sessions and retained Themelio.
Separate ACL users and passwords enforce pharos:auth:* and themelio:* key
prefixes. Commands are explicitly allowed; global flush, key enumeration,
scripting, configuration and ACL administration are denied. Persistence uses AOF
with appendfsync always and noeviction. The retained public cache-clear helper
returns HTTP 410.
Three caches remain separate: Argo's controller cache, GitLab's upstream Redis state, and Pharos' ephemeral realtime history. GitLab's upstream key/command contract has no qualified shared prefix. Realtime history is disposable, while session revocation needs durable writes; controller recovery remains independent of application storage. The resulting counts are one PostgreSQL, one RabbitMQ, and four Redis/Valkey servers.
Installation and verification
Ordinary make deploy converts a recognised personal dev-box installation once.
It discards the old application data, including GitLab repositories and uploads,
and installs a consistent new set of databases, volumes, buckets and credentials.
This applies to the previous per-service providers and the intermediate shared
PostgreSQL 17 layout. PostgreSQL 18 uses a persistent /var/lib/postgresql mount
and versioned PGDATA; GitLab 19.5.1/chart 10.5.1 supports the selected server.
Its upstream Gateway API installation remains disabled because the platform
catalogue owns ingress.
Before cleanup, a versioned checkpoint records the saved target, machine, node,
disk, ownership record and application/storage UIDs. The root and all children
are suspended and checked for active operations. Cleanup deletes only the
captured application and namespace identities, waits for normal volume/bucket
finalisers, then durably advances to installing. An interrupted retry resumes
from that phase. Once installation has started, deploy cannot clean it again.
Argo owns all recurring application and provider reconciliation.
The VM, Kubernetes, Ceph, dedicated disk, certificates, build cache and independent
laboratory remain in place. Takeover leases and unrelated maintenance block
conversion. After Argo readiness at the delivered revision, storage and routes succeed,
the checkpoint becomes ready and ordinary deployment clears its maintenance
record. Later deployments preserve application data, PVCs and credentials. An
explicit reviewed reset remains available through the app and Make.
The deterministic verification lane renders all consumers, checks provider aliases and credential boundaries, and exercises negative fixtures for direct broker management and administrator credential leakage. Provider qualification must also prove successful own-scope access and rejected cross-consumer access, then exercise a fresh deployment and the Colima takeover/hand-back workflow.
Argo-started operations
Amended 17 October 2026. Argo CD starts every sync operation. Delivery applies the root Application and observes; it does not request child operations, retry them or order them. A child is ready when it is Synced and Healthy at the delivered revision, with no queued, running or failed operation at that revision (decision 0039). The CLI gate, the setup agent and Argo's own child health assessment use this one rule. Argo's self-management is the root's first wave, so a changed health rule applies before Argo assesses the later waves.
A revision that leaves a child's rendered desired state unchanged needs no new
operation, so its administration hooks do not run again. Inputs that hooks read
from outside Git are therefore part of desired state. The root passes one
checksum of the private AAI overrides, public issuer trust and administration
image digest to
pharos-aai; a change makes it OutOfSync, and Argo reruns its administration
hooks in wave order. The administration image contains the hook code and default
client registry. Completed hook Jobs are excluded from normal Argo comparison,
so an image override alone does not trigger their rerun. Route inputs and runtime
image digests change ordinary resources directly.
Child Applications retry failed operations without a count limit, with bounded backoff, so Argo recovers once a provider becomes available. Administration hooks wait for the providers they use rather than relying on external ordering. An earlier delivery's still-running operation may be ended so that Argo starts the delivered revision.
One generator for shared-provider credentials
Amended 18 October 2026. The infrastructure credential Job is the only generator
of the credentials it gives a shared provider and its consumers. It keeps them in
one record in pharos-infra, read only by that Job. Every consumer Secret it
publishes is a copy of that record: a missing or changed copy is written again from
the record on the Job's next run, and the database setup it derives sets the role
passwords from the same record. Consumer administration hooks wait for these
Secrets and never create them; they generate only credentials that their own
service alone uses. A lost record is recovered from the existing copies; copies
that disagree with each other require a clean installation.
An existing record missing required fields stops the Job before any write. Restore
the complete record before retrying; missing fields never trigger password replacement.
References
- GitLab PostgreSQL requirements
- Keycloak database support
- RabbitMQ virtual hosts
- Waldur 8.1.2 management backend
- Redis ACL semantics
Page source
Source snapshot · Revision f928834c8.