For AI agents: the complete documentation index is available at /llms.txt, the full documentation bundle is available at /llms-full.txt, and this page is available as Markdown at /reference/docs/development.md.

Developer guide

Start here to set up your personal dev box and work on Pharos. Run the commands below from the repository root.

Your dev box is a Linux VM running the platform and its databases through Argo CD. Your workstation holds your checkout and editor. Takeover runs selected services on your workstation while they use the dev box's dependencies; saving a file reloads the affected service. Hand-back returns those services to the VM. Pharos DX gives you the same controls in a Workspace window and tray.

CLI at a glance

Once your .env and dev box are ready, the daily loop uses three commands:

CommandWhen to use it
make takeoverRun changed services locally; saving reloads them
make handbackReturn those services to the dev box
make reconcileDeploy your committed changes; an active takeover resumes afterwards

Setup and recovery are occasional actions:

CommandWhen to use it
make devbox-setupSet up or repair your dev box with checks and progress
make deployApply the full stack, including host and cluster configuration
make devbox-resetReview a clean reinstall and type the target name to confirm data removal

make devbox-setup wraps the full deployment with a destination review and structured progress, as used by the app. Choose it for guided setup or recovery.

make deploy is idempotent: repeating it converges on the committed configuration and preserves application data on the current layout. It still runs provisioning and readiness checks. make reconcile is the quicker path for normal application changes and matches Deploy this commit in Pharos DX. The one-time conversion from the old layout discards old application data; see setup.

Sign-in uses local Keycloak and test users by default. To use hosted AAI, open Dev box → Sign-in provider in Pharos DX, enable Use external AAI, select its issuer and private client file, then save and deploy. CLI users can run make aai-template, fill the private file, set AAI_MODE=external and its public issuer/path in .env, then run make aai-check and make reconcile. Provider setup lists the complete steps and required registrations. Keep client secrets outside .env.

Secondary commands: diagnostics and explicit plans

These are useful for inspection and automation. Pharos DX handles the equivalent status and review steps in its UI; CLI developers can use them when needed.

CommandPurpose
make urlsConnect browser access and list application and administration URLs
make credentialsList service accounts; use COPY=<id> or SHOW=<id> for one password
make support-linkLink the local operator to Zammad so Support can be tried (PROFILES=zammad); see the Support README
make doctorInspect workstation and dev-box health
make takeover-statusInspect takeover ownership and source compatibility
make takeover-logs SERVICE=betoRead a service's logs
make devbox-planPreview setup without installing; setup also creates its own plan
make devbox-plan MODE=resetPrepare a reset review for non-interactive execution
make devbox-reset PLAN=<id> CONFIRM_TARGET=dev-yournameExecute that reviewed reset

In a terminal, make devbox-reset presents the plan and asks for confirmation itself. Explicit plan IDs are only needed when separating review from execution, such as in automation. make help lists the commands and optional tools.

1. Prepare your workstation and VM

You need Git, Make, SSH and uv on your workstation. Install kubectl and Helm for takeover, and the Node/pnpm versions listed in the repository prerequisites for frontend development and repository checks.

Choose where your local services run:

ChoiceWhat to prepare
Docker on macOSDocker Desktop or Colima; save its socket in .env below
Docker on Linux / WindowsRootful Docker Engine, not the snap package; on Windows, keep the checkout and tools inside WSL2. Containers reach the dev box through a session relay container, so host firewalls need no Docker bridge rule; keep traffic between containers on the default bridge enabled (Docker's default)
NativeGo for Go services; uv prepares the frozen Metallo and Themelio Python environments, and the controller prepares pinned Node/Corepack for web. Run make takeover-setup, then the administrator command it prints, once per workstation

Native takeover runs application code and dependencies with your user account's filesystem permissions. Use it only with trusted code: clearing inherited credential variables and choosing a private application home still leaves files accessible wherever your user can read them.

Prepare your own Debian 12 AMD64 VM with systemd, Git, Python 3, Make, Bash and rootful Docker Engine. The SSH user needs passwordless sudo. The tested VM shape is 16 vCPU, 32 GiB RAM, an 80 GiB root disk and a separate empty 100 GiB data disk for Ceph. Setup installs the platform on this VM; it does not create the VM. Follow the runner preparation before first setup.

On a fresh Debian 12 VM, install the prerequisites with:

sudo apt-get update
sudo apt-get install -y git python3 make bash ca-certificates openssh-client util-linux docker.io
sudo systemctl enable --now docker

If Docker Engine is already installed, omit docker.io to keep that installation. Setup reports missing tools with these manual commands; Pharos DX displays the same notice. Run the commands on the VM, then repeat the dev-box checks.

Check access from your workstation:

ssh your-vm-alias true

Colima file sharing

On macOS, Colima can provide the Docker engine for takeover. Start a profile with home-directory sharing so it can access your checkout and Telepresence files:

colima start pharos --cpus 8 --memory 12 --disk 50 --mount "$HOME:w"

Use unix:///Users/yourname/.colima/pharos/docker.sock as DOCKER_HOST below, replacing yourname with your actual macOS home directory. This also requires your checkout to be under your home directory. Docker Desktop can use unix:///Users/yourname/.docker/run/docker.sock instead.

2. Save your workstation settings

If the checkout has no .env yet, copy .env.example to .env. Otherwise edit the existing file. Set these five values for your workstation and VM:

RUNTIME=docker
DOCKER_HOST=unix:///var/run/docker.sock
TARGET=dev-yourname
SSH=your-vm-alias
STORAGE_DEVICE=/dev/vdb
  • RUNTIME: docker or native.
  • DOCKER_HOST: your local Docker socket; use the macOS path above when applicable.
  • TARGET: your personal dev-box name, starting with dev-.
  • SSH: the SSH alias that reaches your VM.
  • STORAGE_DEVICE: the VM's empty dedicated data disk. Verify the actual device; the example /dev/vdb must never point to the OS disk or a disk containing data.

Use unquoted KEY=value lines and absolute paths. Shell variables such as $HOME are not expanded in this file. .env is private and ignored by Git. Retrieve application passwords through Credentials or make credentials.

All Make examples below use these saved settings. Override a value for one command when needed, for example make takeover SERVICE=beto. With no runtime setting the default is native; the default Docker socket is unix:///var/run/docker.sock.

Install Pharos DX

The desktop app is optional; every workflow below has a Make command. Installing a packaged app requires no Rust or frontend build.

macOS preview limitation: installers currently lack Developer ID signing and Apple notarisation. First launch may require Privacy & Security → Open Anyway. If that screen or approval is unavailable, use the Make commands until you can approve the app or a signed, notarised build is available. See the macOS limitation and launch steps.

  1. Sign into GitHub with repository access and open Pharos DX releases. Choose the newest dx-v…-preview.… release and download your installer from Assets: macOS universal .dmg, Windows x64 .exe, or Linux x64 .deb / .AppImage.
  2. Install and open Pharos DX. On macOS, drag it into Applications; on Windows, run the installer; on Debian/Ubuntu, install the .deb with your package manager. Follow the OS-specific installation steps, including unsigned-preview limitations and Windows/WSL2 requirements.
  3. In Settings, enter your checkout's absolute path and choose Connect workspace. On Windows use its WSL2 Linux path. The app reads the .env you just saved. Open Dev box to continue.

Releases include checksums and remain available beyond CI artifact expiry. Updates are manual: finish active operations, quit the app and install the newer release. See installation and update details.

3. Set up your dev box

In Pharos DX: connect your checkout in Settings, open Dev box, check the target, SSH alias and storage disk, then choose Check dev box followed by Set up dev box. Progress follows the actual installation stages. You are ready when the app reports All checks passed. The first check loads the installed Optional services; change them, then run the checks again before setup.

In a terminal:

make devbox-setup

Use make devbox-setup PROFILES=zammad to replace the selection; several names may be comma- or whitespace-separated. PROFILES=none starts a reviewed removal. Direct make deploy may add profiles and refuses to remove an installed service.

Setup checks the destination and installation scope, prints its plan, installs and waits for healthy services. On success, it prints the application, administration and installed laboratory URLs automatically. Browser access is already connected; use make urls whenever you want to see the list again. Use make devbox-plan separately if you want to inspect the scope before installing.

Setup delivers your current Git commit. Commit changes that should be installed before starting. A first installation can take about an hour when Waldur has no cached database seed. Keep the operation running while migrations and readiness checks complete.

Upgrading an older dev box: the first setup or make deploy converts its previous infrastructure layout and discards its application data, including repositories and uploads. This happens once; subsequent setups and deploys preserve the new data. The setup review displays the conversion before starting. See the conversion details.

For the desktop app, see preview packages and source builds. make dx starts it from source and requires the Rust/Tauri prerequisites in that guide. The terminal flow works independently of the app.

4. Edit, take over and save

Open your editor and change an application source file. In Pharos DX choose Take over changes, or run:

make takeover

The watcher finds changed services and runs them locally. Save again to reload: Go rebuilds, Python reloads and Next.js refreshes. Other services and shared infrastructure continue running on the VM. The Workspace and tray show which services are local, their health and any compatibility problems.

To work on a particular service before editing, select Run only this service locally in the app, or use:

make takeover SERVICE=beto

Available services: api, realtime-gateway, web, metallo, themelio, beto and compat-web. web is the new platform shell; compat-web is the retained Themelio frontend. An explicit selection replaces the current selection.

ActionPharos DXTerminal
Check local and remote stateServices, traymake doctor
Check takeover and commit compatibilityCompatibilitymake takeover-status
Deploy your committed changesCompatibility → Deploy this commitmake reconcile
Read a service's logsSelect service → Capture service logsmake takeover-logs SERVICE=beto
Finish local workHand back all servicesmake handback

Open the application

Setup and deployment prepare browser access automatically. Run make urls to connect or reconnect and list the platform links. In Pharos DX, select a row in Your services and click a URL in Service details. The app prepares access; administration links require hand-back and open the operator connection. No SSH terminal needs to stay open. This works on macOS, Linux and Windows/WSL2. After a restart or VPN interruption, repeat the command or click to reconnect. Use Disconnect browser access or make devbox-disconnect when finished. The list describes configured routes; use make doctor to check service health.

The deployed platform shell and its takeover replacement are at https://platform.localtest.me; the direct takeover listener remains available at http://localhost:28000. Retained Themelio is at https://themelio.localtest.me. The first frontend compilation can take a few minutes.

Enable the short HTTPS addresses once per workstation session:

make devbox-browser-setup

On an existing dev box, first run make deploy from this version to install the route, certificate and sign-in callback. The command connects browser access, authenticates the current public CA over SSH, checks the deployed route and requests approval for local port 443 and missing certificate trust. It verifies trust afterwards; repeat runs reuse it. On Windows, run it inside your configured WSL2 distribution. The short address stays in the browser; shared AAI sign-in returns to it. Browser visits to the old address redirect here. Existing Git/LFS remotes keep working on either address. See short-address setup for port conflicts, stopping the helper and reconnecting.

If your browser marks the short address insecure, run make devbox-browser-setup. It uses the selected target from .env; no passwords or private CA keys are copied. On macOS, approve the Keychain prompt to trust the CA for SSL in your login keychain. If macOS blocks the prompt, open .local/local-aai/<target>/ca.crt in Keychain Access, expand Trust, set Secure Sockets Layer (SSL) to Always Trust, then close and approve. Repeat the command to verify, and restart a browser retaining an old warning. On Debian/Ubuntu and Fedora, setup installs the public CA in system trust and Chromium/Firefox NSS databases; it installs the NSS tools if missing, with sudo approval. Windows/WSL2 setup requests approval for the Windows user's root store and configures WSL CLI trust. Custom or sandboxed browsers may require importing the same CA through their certificate settings. Setup also configures Git certificate trust. Unrelated certificates remain unchanged. See Apple's trust instructions and Chromium's Linux certificate stores.

The same setup enables https://platform.localtest.me, https://gitlab.localtest.me and https://waldur.localtest.me. For https://ceph.localtest.me, https://argo.localtest.me and Waldur /admin/, hand back any takeover and run make devbox-forward; these routes require operator access. make urls lists the short addresses and their access requirements. Existing Git/LFS remotes remain valid.

Open Credentials in Pharos DX, or run make credentials to find the account you need. Use Copy or make credentials COPY=waldur.staff to retrieve one password. Retained Themelio uses shared AAI; the new platform shell uses pharos.operator. See credential access for database accounts, administrator passwords and terminal reveal. See browser access and login for GitLab, Waldur and operator access. Takeover's public-only HTTPS tunnel denies Argo CD, Ceph and Waldur administration routes. Hand back before opening an operator connection with make devbox-forward; the next takeover switches it back to public-only access automatically. Existing dev boxes need one full setup/deploy to install this separation; see the upgrade steps.

Seed development data

A new dev box has no Projects, groups or repositories. make seed (or make waldur-seed) creates Projects of the Pharos customer for alice and bob: A and B with an LLM order each, Cloud with a cloud order and HPC with an HPC order, and the cloud and HPC offerings if they are missing. Source Control then makes each Project's GitLab group under pharos-projects, and make gitlab-seed adds the sample Git/LFS repository to Project A's repos group, where Themelio shows it; run it again if Source Control has not made the group yet. SEED=<file> applies your own file. Both create only what is missing and are safe to repeat. Waldur members without an account sign in once through the local AAI. File formats: DD-0060.

5. Finish work or deliver a commit

To end a development session, use Hand back all services or make handback. Closing the app, terminal or Tilt leaves the takeover session running.

To install your changes on the VM, hand back and commit your intended changes. In Compatibility, choose Deploy this commit, review the target and commit, then confirm the deployment. Progress follows image builds and Argo health checks; Compatibility refreshes when it finishes. The app deploys the selected checkout's commit, so switch branches or pull changes before starting if needed.

Desktop-only and Markdown-only changes usually need no deployment. The app and make reconcile compare the image inputs and check that every Argo application is healthy and synced. They then report Already up to date without transferring source, rebuilding images or rerunning hooks. The dev-box commit remains the last actual deployment; a different workstation commit is expected here. Changed runtime/deployment inputs, unknown comparisons or unhealthy applications use normal delivery.

The equivalent terminal workflow is:

git add <your-files>
git commit -m 'feat: describe your change'
make reconcile

make reconcile delivers the committed source and builds changed images. Use make deploy when changing host configuration or the cluster foundation. Delivery cannot run during a takeover, so both commands hand back this checkout's takeover of the target first. After a successful delivery they take over again with the same runtime and selection: an explicit SERVICE, or the services that still differ from the new commit. After a failed delivery the services stay on the dev box, and the command prints the make takeover command to resume. Migrations, contracts and deployment changes need a committed delivery before takeover; ordinary application edits can reload locally. The app still requires hand-back, and checks for changed source and expired reviews before delivery.

Recover or start again

SituationNext step
App deployment was interruptedRead Activity, fix the cause, then retry Compatibility → Deploy this commit or make reconcile; both check the target again before recovering the failed delivery
Setup/reset terminal was closed or killedRepeat the same make devbox-setup or make devbox-reset command from the same checkout to reconnect to its progress. In Pharos DX, choose Check dev box on the matching tab
The background worker or a direct make deploy stoppedInspect the private log and any delivery still running on the SSH runner, then review setup again or repeat make deploy once the runner is idle
Takeover was interrupted, or a taken-over service stoppedRepeat the same make takeover command; it restores the broken service and takes it over again
Hand-back was interruptedRepeat make handback
Checkout and dev-box deployment are incompatibleCommit the intended changes, then run make reconcile, or hand back and use Compatibility → Deploy this commit
Dev box needs its platform configuration reappliedRun make deploy
You want to discard application data and start cleanUse the reviewed reset flow below
Another takeover owns the dev boxCheck its owner in Compatibility or make takeover-status; the owner must hand it back before delivery
Disk space is lowCheck workstation, Docker/Colima and VM disk usage in make doctor before retrying

In Pharos DX: hand back, open Dev box → Reset, choose Check dev box, then Review reset. Read the scope, acknowledge permanent data removal and choose Reset & reinstall.

In an interactive terminal: run make devbox-reset, review its plan and type the target name when prompted. Reset removes application databases, repositories, messages, sessions and uploads, then reinstalls. It keeps your VM, SSH access, Kubernetes, Ceph, certificates, build cache and local source. See reset scope and recovery for non-interactive commands and interrupted operations.

SSH runner delivery

Setup, reset, make deploy and make reconcile use a Linux SSH runner on every workstation OS. The default runner is the dev box itself. Deployment does not require Docker on the workstation; Docker takeover still needs its local engine.

Prepare the runner once with Git, Python 3, Make, Bash and rootful Docker Engine. Its SSH user needs passwordless sudo. Setup and deployment automatically prepare dedicated target SSH credentials remotely, including when the runner targets itself. Run everything from your workstation using its existing trusted SSH connection. Missing dependencies show an install command before credential creation. RUNNER_SSH_DIR in .env selects a manually managed dedicated directory; personal SSH directories are protected. The alias must resolve to a VM address reachable from Docker bridge containers; do not use 127.0.0.1 or localhost for self-access. The API tunnel runs in a separate container, where loopback addresses refer to that container.

For an existing installation made from another checkout on the runner, pass REMOTE_ROOT=/home/your-user/pharos-platform on the first make deploy to reuse its private state. Omit this for a new installation. The runner binding is saved; DEPLOY_RUNNER=your-linux-alias can select a separate runner for delivery.

Keep the Linux runner dedicated to development and free of unrelated credentials. Deployment tools administer its rootful Docker Engine and use passwordless sudo. These capabilities give them broad control of that host, including its files; individual SSH mounts limit routine exposure within this existing trust boundary.

An explicit runner must match any existing saved runner binding. Setup and ordinary deployment use the same selection: setup inspection reads the saved delivery identity from the selected runner, and the dev box checks it against its live node identity before any change. The runner calls the internal Linux deployment entry point directly, without another SSH delivery dispatch. Only committed source is transferred, with Git history reuse. Browser trust and test-user credentials return privately to .local/local-aai/<target>.

Optional private AAI inputs and cached Waldur seeds must be prepared in the runner's private state; the source bundle does not transfer them. Reconcile takes the ordinary delivery path when private AAI overrides exist, even if committed image inputs match; a changed override reaches Argo as part of AAI's desired state.

Further reading

Tilt dashboard

Tilt is an optional alternative dashboard. Install Tilt 0.37.8, run make tilt and open http://127.0.0.1:10350. It uses the same takeover flow and provides hand-back, delivery and log actions. Closing Tilt leaves takeover running; use make handback when finished.

Acceptance checks

The repeatable acceptance commands exercise reload, browser login, connection loss, hand-back and Git/LFS.

Readiness

See runtime qualification results for the tested commits and environments, including Colima on macOS, and layout conversion qualification for the shared-infrastructure rollout.

Page source

Source snapshot · Revision f928834c8.