#DD-0060 — Development seeds for Waldur, GitLab and SSH Access

Related ADR: ADR-0060.

Status: Draft for review, with dev-box evidence on Waldur 8.1.3 and GitLab 19.4.2.

Date: 2026-10-22.

#Use

make seed                        # the Waldur seed, as below
make waldur-seed                 # dx/common/waldur-seed.yaml
make gitlab-seed                 # dx/common/gitlab-seed.yaml, after make seed
make waldur-seed SEED=my.yaml    # another file
make ssh-access-seed             # dx/common/ssh-access-seed.yaml
make ssh-access-seed CHECK=1     # fail instead of writing on drift

The commands use the target and SSH alias of the root .env, like make credentials. They print one line per seed entry, created or present, and a total. A repeated run prints only present lines and sends no change.

Members must be users of devops/aai/users.json. Waldur names a user after the subject of the sign-in, so the Waldur seed looks members up by the subject the local AAI gives them. When a member has no account in the service yet, the command changes nothing, signs that member in once with dx/common/scripts/service-login.sh and applies the seed again. The sign-in needs the short addresses (make devbox-browser-setup) or PHAROS_SERVICE_EDGE=1. The GitLab seed names no members: a Project's members reach its GitLab groups through Source Control, once they have signed in to GitLab.

#Seed files

Each file has a JSON schema beside it (waldur-seed.schema.json, gitlab-seed.schema.json). The command validates the file before it connects.

# dx/common/waldur-seed.yaml
version: 1
customer: Pharos                 # must exist; the installation creates it
projects:
  - name: Project A              # matched by exact name within the customer
    description: Development project A
    members:
      - {user: alice, role: manager}   # admin | manager | member
    orders:
      - offering: llm-access-offe      # provider offering slug
        plan: LLM access plan          # optional when one plan is active
        name: project-a-llm            # resource name; matched to skip a repeat
        attributes: {models: [qwen2.5-0.5b-cpu]}
        limits: {max_budget: 1, tpm_limit: 100000, rpm_limit: 100}
  - name: Project Cloud            # a cloud order only
    members:
      - {user: alice, role: manager}
    orders:
      - {offering: cloud, name: project-cloud, limits: {cpu: 2, memory: 4, storage: 5}}
  - name: Project HPC              # an HPC order only
    members:
      - {user: bob, role: manager}
    orders:
      - {offering: hpc-icarus, name: project-hpc}

The offerings section lists the cloud and HPC offerings under the identity of their owners: the Site Agent's installation hook registers cloud and the Icarus laboratory (make metallo-icarus) registers hpc-icarus. The seed finds an offering by slug and does not change it, so it never duplicates an installed one, with two exceptions: it adds a plan to an offering that has none (Waldur activates only an offering with a plan), and it activates one that is still a draft; a paused or archived offering is left as an operator set it. It creates the provider organisation, service provider, category, offering (with the components its orders limit: the cloud offering's cpu, memory and storage, which Waldur refuses an order without), plan and activation only when the offering is missing. An offering whose slug belongs to another organisation stops the run. Keep the section in step with its owners, because a copy is only used on a dev box where they have not run. An order names its offering by slug.

The seed and the owners can run in either order: the provider names and slugs are the owners' (Pharos cloud provider, pharos-cloud; Icarus-hpc provider), and each owner finds an existing organisation, offering, category and plan by them. make metallo-icarus also brings the HPC offering's cpu component and plugin options to its own values. On a box where make metallo-icarus has not run, nothing processes the HPC order of Project HPC, so it is placed and approved but never completed.

# dx/common/gitlab-seed.yaml
version: 2
root: pharos-projects            # optional: Source Control's root group, this by default
lfs:                             # LFS files added to every repository: path -> bytes
  weights/tiny.bin: 1048576
projects:
  - project: Project A           # a Project of the Waldur seed, by its exact name
    repositories:
      - kind: repos              # models | datasets | spaces | recipes | upskillings | repos
        name: sample             # receives dx/common/gitlab-seed/sample plus the LFS files

The GitLab seed adds repositories to the groups Source Control made for the seeded Projects, at <root>/<Project ID>/<kind>/<name>, where Themelio reads them. It finds a Project's group among the subgroups of the root by the name Source Control gives it, <Project> (<Project ID>), where the Project ID is the group's path, and then the kind subgroup. The schema refuses a Project name that Source Control would rewrite in that name. All groups are found before anything is written: while a Project or kind group is missing, or two groups carry the same Project name, the run changes nothing and names the Project. Source Control makes the groups on its next pass after Beto takes the Project from Waldur, so a first run straight after make seed may have to be repeated.

Every GitLab repository receives the text files of dx/common/gitlab-seed/sample: a README, a CSV, a Python script and a .gitattributes that tracks *.bin and *.safetensors in LFS. The LFS files' bytes are generated deterministically on the dev box, so the platform repository stores no binaries and every run produces the same object IDs. LFS paths must end in .bin or .safetensors and be at most 100 MiB.

#SSH access browser fixture

# dx/common/ssh-access-seed.yaml
version: 1
site: hpc                      # the single Site of the Site API and of SSH access
currentAup: v1
projects:
  - {id: project-a, name: Project A}
members:
  - user: alice                # a user of devops/aai/users.json
    member: 00000000-0000-4000-8000-0000000a11ce  # Party member stand-in; keep it stable
    uidNumber: 5000001         # the UID the laboratory's Waldur pool assigned
    revision: 3
    acceptedAup: v1
    projects: [project-a, project-b]
actors:                        # actors that are not AAI sign-ins: operator and CLI fixtures
  - {actor: 00000000-0000-4000-8000-000000000001, member: 00000000-0000-4000-8000-0000000a11ce}

The SSH access fixture stands in for Beto's Party adaptors until they replace it. The browser API acts as the Party of the signed-in session, and a Party appears at a person's first sign-in: services/api/adaptors/authentication_sessions.go calls PartyID the subject the SSH context sees. So the fixture needs an actor entry keyed by that Party, not by the subject the local AAI derives from the user name. Without it every SSH access route answers 403 member_not_found and the Site list is empty.

The Party is created per installation, at the first sign-in, so it cannot live in the committed manifests. make ssh-access-seed reads it, read-only, from pharos.party_authentication_identities on the dev box selected by the root .env (SELECT only; nothing is written there), keyed by the subject the local AAI gives the user (uuid5(NAMESPACE_URL, 'pharos-lab:pharos:' + user)). A user who has not signed in yet has no Party: the seed writes the rest, prints that the SSH access page will answer member_not_found for them, and asks for a sign-in and a second run. The delivered manifests therefore carry only the actors that are not sign-ins; a dev box's run adds the Party entries of the users who have signed in.

make ssh-access-seed validates the file, takes each member's posix_username from devops/aai/users.json and the Projects' names from the seed, and renders one owners.json value with one member per seeded user, one actor per sign-in Party and the actor fixtures of the seed. It writes that value into gitops/manifests/pharos/base/ssh-access/workload.yaml, which make deploy delivers. The ConfigMap belongs to the pharos Argo application, which self-heals it, so the seed and the manifests are the only writable path: nothing edits the running ConfigMap or a database by hand. A second run reports present for every member and changes nothing; CHECK=1 exits non-zero when the delivered manifests differ from the seed. NO_PARTIES=1 renders the form the repository carries, without reading a dev box.

The seed never guesses a UID: uidNumber must equal the POSIX identity the laboratory's Waldur pool assigned (dx/metallo/icarus-hpc), because the Site agent refuses to write a key for a member whose UID disagrees with the directory entry. revision follows the member's Site identity: bump it when the username or UID changes.

#What is created and when an entry counts as present

The seed never changes an existing entry. A different role, description or limit on an existing entry stays as it is. A repository that exists without branches, after an interrupted run, receives the sample on the next run.

GitLab repository content is written without Git: the toolbox image has none. The seed uploads each LFS object through the Git LFS batch API first, which links it to the project, and then commits the text files and LFS pointers through POST projects/:id/repository/commits. A clone with Git LFS restores the objects.

#Credentials and process

• Workstation: dx/common/scripts/dev_seed.py validates the seed, adds the GitLab sample files and sends one program over SSH (credential_agent.py, seed_agent.py, the service's agent and its REST program). SSH runs without agent or port forwarding, as make credentials does.
• Host: seed_agent.py checks that the cluster's node UIDs match the target's binding and that the platform is not under maintenance. Any failure crosses SSH as a fixed message, never as subprocess output.
• Waldur API container: waldur_seed_agent.py runs one bash -c that mints a token for the staff user pharos-seed (token_lifetime 900 seconds) with waldur shell, runs waldur_seed_api.py with the token in its environment and deletes the user's tokens afterwards.
• GitLab toolbox container: gitlab_seed_agent.py runs one gitlab-rails runner that creates a root personal access token named pharos-seed (scope api, expiry one day, the shortest GitLab allows), spawns gitlab_seed_api.py with the token in its environment and revokes the token in an ensure block. The program first revokes any active pharos-seed token left by an interrupted earlier run. It reaches GitLab at gitlab-webservice-default.gitlab.svc:8181. LFS upload addresses name the external URL, so the program sends them to the same path on the internal address.
• Reports: the REST programs print one JSON line with the entries' labels. A rejected request reports the HTTP status and the names of the rejected fields, never the values or the response body.

#Verification

• dx/common/tests/test_waldur_seed.py:

  • checks every Waldur request body against the 8.1.3 OpenAPI schemas extracted to dx/common/tests/fixtures/waldur-8.1.3-seed-requests.json;
  • checks that the default seed applies once and a second run sends no change;
  • checks that missing members, customers and offerings change nothing and name the entry;
  • checks that invalid seeds and unknown users are refused.

• dx/common/tests/test_gitlab_seed.py, against a GitLab double that holds the groups as Source Control makes them (it checks the seed's logic, not GitLab):

  • checks that the default seed adds its repository to the Project's kind group once, writes no group or member, and that a second run sends no change;
  • checks that only the group named after the Project and its own ID is used, and that a missing root, Project group or kind group, or two groups for one Project, change nothing, not even a token;
  • checks that the earlier group form, members, unknown kinds and Project names Source Control would rewrite are refused, and that the default Project is one the Waldur seed makes;
  • checks that LFS objects are uploaded before the commit that names them and that only an earlier run's token is revoked;
  • checks that the token lives in one Rails process and is passed to Python in its environment.

• dx/common/tests/test_ssh_access_seed.py:

  • checks that the delivered ssh-access-fixture ConfigMap is exactly the rendered seed, and that a second run changes nothing;
  • checks that a signed-in user is keyed by the Party their first sign-in created, with her posix_username, UID and Projects, that a changed Party replaces the old actor, that a user without a sign-in is reported and keyed by nothing, that the fixture never keys an actor by an AAI subject, and that the operator actors still name their member;
  • checks that unknown users, undefined Projects, a shared member identifier, an actor without a member and an out-of-range UID are refused, and that the drift check fails on a changed manifest.

• services/metallo/tests/ssh_access/test_browser_fixture.py reads the delivered ConfigMap through the real fixture adaptor and checks the same members, the Site binding, the Projects, the AUP answer, that a Party actor reaches its member, that no actor is an AAI subject, and that a stranger is no member. services/metallo/tests/test_icarus_hpc.py keeps the end-to-end lane pinned to alice's revision.

• Dev box dev-ikons, 22 October 2026:

  • Waldur 8.1.3: a seed with one Project, a member and an LLM access order created all three, and the site agent completed the order. A second run created nothing, and no pharos-seed token remained. The test Project and its Slurm accounts were then removed.
  • Waldur after the GitLab change: a throwaway seed was applied twice (2 created, then 2 present), and its Project was deleted.
  • GitLab 19.4.2, first run: 4 entries were created in 1 min 48 s, after bob signed in once. A second run created nothing, in 55 s.
  • GitLab as bob (Developer): with an ordinary HTTPS Git client and Git LFS, bob cloned playground/sample. The LFS file was restored with the expected SHA-256. A push to a new branch with a new LFS file succeeded, and a push to main was refused.
  • GitLab tokens: no active pharos-seed token remained.

  These GitLab runs used the earlier playground form, a top-level group with its own members.

Not covered:

• The Project form of the GitLab seed on a dev box: a repository in a Source Control Project group, shown in Themelio.
• Waldur 8.1.2.
• GitLab versions other than 19.4.2.
• A seed with many groups or large LFS files.

#Limits and later work

• The Rails start in the GitLab toolbox takes one to two minutes on a loaded dev box, and the toolbox requests only 50m CPU.
• The seed finds a Project's group by the name Source Control gives it (group_name in services/metallo/contexts/source_control/domain/gitlab.py); a change to that name must change the seed. It creates repositories through GitLab's API because Source Control has no command that creates one; when it gains one, the seed should use it.
• A first run straight after make seed can find no group yet: Source Control makes it on its next pass. The run changes nothing and says so; run it again.
• Mirroring the retained Themelio's repositories from gitlab.grnet.gr is not included. It needs a personal token on that server and can transfer large LFS objects.