#0055 — Member SSH access to Icarus-hpc

Status: Proposed. Date: 21 October 2026. The implementation is in DD-0055. Deciders: ikons; review by Metallo owners (dkatsimpris) and the Icarus-hpc administrators (jimsiak).

This decision builds on the September prototype (#137, Approach C). It keeps the prototype's fingerprint identity, lock ordering, withdrawal handling and compensation for an add that lands after its withdrawal. It moves key ownership from the Waldur offering user to the member, in a Metallo context, and delivers changes to Icarus-hpc through the Site API instead of a CSV queue.

#Terminology

Thick arrows are outbound HTTPS connections from Icarus-hpc; each agent uses its own credential.

#Context

GitHub #678 defines member-facing SSH key routes: gates, a stable key ID across resubmission, per-member fingerprint uniqueness and the states pending_deployment, deployed, rejected, pending_revocation and revoked. Keys belong to the member, not to a Project or Waldur offering. Each member has one native account on Icarus-hpc.

Upstream Waldur 8.1.3 offers per-user key storage (/api/keys/) and a provider key listing, but no per-key state, comment or result reporting. Writing a key for a member needs that member's own Waldur credential. Key events reach only staff, support or the member.

The September prototype is a Waldur extension. A member files a request to add or remove a key, the HPC side approves or rejects it, and the extension keeps the list of approved keys for each offering user.

#678 was written afterwards and describes the same lifecycle from the key's point of view. A pending add request corresponds to pending_deployment, an approved add to deployed and a rejected add to rejected. A pending remove request corresponds to pending_revocation, and an approved remove to revoked.

#678 also adds a few things. A key keeps one ID for its whole life, so a revoked key can be resubmitted without pasting it again; in the prototype each request has its own ID. #678 checks the AUP and the HPC username before accepting a key, and lets the page validate a key without submitting it.

The main difference is scope. The prototype attaches keys to an offering user, that is, to one member in one offering. SSH keys are personal, so this decision attaches them to the member, across all their Projects and offerings.

On Icarus-hpc, the Site agent's account-name plugin creates each member's POSIX entry under ou=pharos under the username Metallo wrote from the Party's site binding, and pins the UID it got on the offering user. The offering's policy is service_provider; Waldur itself does not derive the name from a claim. The LDAP schema is openssh-lpk. The Icarus account model is proposed in PR #643.

The Icarus-hpc administrators confirmed on 21 October 2026:

• authorized_keys is used only for specific local users, so LDAP is the only key source for Pharos members;
• their own key-sync script writes elsewhere, not under ou=pharos, and Pharos UIDs come from a separate range;
• when a Waldur project ends they remove the Slurm association; the user can still log in with SSH but cannot submit jobs;
• the access agent can run on the same host as the Site agent; outbound HTTPS from that host to Icarus-cloud is permitted;
• there is no site-specific key policy beyond LDAP and Waldur defaults;
• SSSD propagation time is not yet known.

On 23 October 2026 they also confirmed:

• Pharos UIDs come from 5000000–5499999 and allocation groups from 5500000–5599999;
• the directory is OpenLDAP without LDAPS;
• no GRNET users are synchronised into the Icarus LDAP, so the Site agent creates every Pharos entry under ou=pharos;
• the 120-second propagation window is acceptable;
• they will apply the directory, service-account and Slurm setup that Pharos specifies.

#Decision

1. Metallo owns member SSH keys. A new Metallo context, ssh_access, owns each member's keys for a Site and their #678 lifecycle in its own PostgreSQL database with dedicated credentials. Waldur does not store these keys. No Waldur extension is installed.
2. The Site agent creates accounts; the access agent writes keys. The upstream Site agent remains the only writer of LDAP entries, POSIX attributes and groups. The access agent never creates, renames, disables or deletes entries. It adds or removes individual sshPublicKey values. It adds the ldapPublicKey auxiliary class atomically with the first key and removes it atomically with the last key, supporting directories where the class requires sshPublicKey (MUST).
3. The LDAP server enforces one writer per attribute. The access agent binds as cn=pharos-ssh-access with write access to sshPublicKey and only the ldapPublicKey value of objectClass under ou=pharos, plus read access to identity and concurrency attributes. All other class values remain read-only to that bind. The Site agent's bind DN loses write access to sshPublicKey; account creation must omit ldapPublicKey when the schema requires a key.
4. The two agents run as separate service identities on the same host. The access agent has its own Unix user and systemd unit. It has no Slurm, Waldur, root or sudo authority, and no platform database credential.
5. The access agent reaches Icarus-cloud only by outbound HTTPS. It pulls work from, and acknowledges results to, the existing Metallo Site API on dedicated /v1/site/ssh-access/* routes on the separate Site API host. The edge routes that hostname and prefix to the Site API while browser API and web traffic keep separate hosts, backends and credentials. It authenticates with its own native service credential: a high-entropy bearer whose digest and grant the Site API retains. Its grant covers only the SSH access routes for its Site. The Site Runtime's grant does not cover them, and the same verifier serves the native Site Runtime.
6. The access agent converges per member. Work is level-triggered: for each member it applies the desired key set to the exact entry identified by uid and uidNumber. Each modification carries a critical RFC 4528 assertion for both uidNumber and the entry's immutable entryUUID, so an account replacement at the same DN cannot receive the previous account's key. Read-back verifies the same identity. A fresh read and an entryCSN assertion also fence concurrent key/class changes; contention retries rather than rejecting a valid key. A missing entry leaves keys pending and retried; it is not a rejection. The agent reconciles after uncertain outcomes.
7. Departure follows Icarus-hpc practice. Leaving every HPC Project removes the Slurm association through the Site agent; the LDAP entry and its keys remain, so the member can still retrieve data. New submissions are refused with not_eligible. Keys are removed only by member revocation or by an AUP change.
8. Beto inputs arrive through Metallo-owned ports. ssh_access reads member Site identity and HPC eligibility through its own ports. Until Beto's Party and Project integration events (feat/beto-party-project-governance) are on main, configured fixture adaptors supply them. Subsequent adaptors consume PartyProviderBindingEstablished, PartyProviderBindingRetired, ProjectChanged and ProjectAccessChanged.
9. The browser contract follows #678 within current platform rules. Routes are declared in the API's OpenAPI contract and reach ssh_access through typed command and query bindings. Browser updates use a typed Protobuf projection and Centrifugo once the page is mounted. There is no browser polling, no NextAuth route placement and no email matching.

#Consequences

Positive: one authority for keys and their states, held per member; Waldur and the Site agent need no Pharos-specific code; native authority on Icarus-hpc is limited to the key attribute and one auxiliary class value and enforced by the LDAP server; ordering between account creation and key submission does not matter; the native credential verifier and Site API route also serve the native Site Runtime.

Negative: keys added in Waldur Homeport are ignored on Icarus-hpc; a new PostgreSQL context and a Site-side process must be operated; deployed and revoked depend on the SSSD cache: the administrators accepted a 120-second window, which the laboratory measured but Icarus-hpc has not yet qualified.

Neutral: the access agent and Site API routes can ship in separate PRs behind the same lifecycle; Beto adaptors replace fixtures without changing the context.

#Open questions

• Context name: ssh_access is proposed; credentials was also considered but overlaps with service credentials and delegated member JWTs.
• Source of the AUP gate: the retained Themelio AUP or the Waldur offering's Terms of Service.
• The access agent uses ldapi:// with SASL EXTERNAL on the directory host, mapped from its Unix identity to its restricted LDAP identity without a password. The upstream Site agent supports simple bind only and uses loopback ldap:// on Icarus-hpc. Confirm that listener and the service-user mapping with jimsiak; the access agent still refuses non-loopback cleartext LDAP.
• The Site agent's bind DN ACL (pending jimsiak). No existing entry needs ldapPublicKey, since no GRNET users are synchronised.
• Per-member key limit (10 proposed) and FIDO (sk-ssh-ed25519) keys.

#Related documents

• DD-0055.
• GitHub #678, #137, #141.
• PR #643: Icarus HPC accounts, member execution and run path.
• Secure Site communication.
• Waldur Site agent LDAP plugin.

Page source

Source snapshot · Revision f928834c8.