#SSH Access

SSH Access is the Metallo context that owns members' SSH public keys for each Site and their #678 lifecycle. Waldur does not store these keys. The access agent on the Site applies them to LDAP through the Site API. See ADR-0055 and DD-0055.

#Aggregate

domain.access.MemberSshAccess is one root per member and Site. Its identity is derived from both, and it doubles as the opaque memberRef the Site sees. Its children are the member's keys (domain.key.SshKey). One root makes per-member fingerprint uniqueness, the key limit (10) and serial submissions atomic with the root lock and version. The root also records the member's last known Site account (username, uidNumber, source revision); an older identity revision never replaces a newer one. A different username or UID requires newer evidence and acknowledged removal of all existing keys. Validation and submission enforce the same account check.

Canonical submissions also record an AccessGrant on each key: the Party's authentication authority version and the exact Site-binding ID. The submission transaction rechecks these against the owner projection under its locks. Private snapshots read older keys without this field; the canonical withdrawal worker revokes such unproven grants once, so the member must explicitly resubmit. HTTP and browser projection shapes are unchanged.

Each key keeps one id for its whole life. Its states follow #678: pending_deployment, deployed, rejected, pending_revocation and revoked. Every submission, resubmission and revocation issues a new generation and asks the Site for the key's presence (present while pending or deployed, otherwise absent). Site results for an older generation are stale and change nothing; a repeated result for the current generation is replayed; a different result for it, or one that contradicts the desired presence, is a site_result_conflict.

An applied or removed result proves the LDAP state only. The key stays pending until max(observedAt, generation issued) + propagation window, then becomes deployed or revoked. rejected is final and carries the Site's reason.

Revoking a pending key that has no applied result also keeps it in pending_revocation. A missing acknowledgement does not prove that an add never reached LDAP, so the next generation asks the Site for absence. Only a removed result and the propagation window make it revoked; delete and resubmit remain unavailable until then.

#Operations

Gate failures raise HpcAccessGated with every reason that holds: not_eligible, aup_not_accepted, aup_outdated, hpc_username_missing. Key checks run in the order private key (key_private, never echoed), format (key_malformed), type (key_type_unsupported), strength (key_too_weak), then the name (key_name_invalid). Gate and key check failures are raised before the transaction and record nothing. Duplicates (key_duplicate, with the existing record), the limit (key_limit_reached) and state conflicts (key_state_conflict) are recorded rejections that change nothing. #678's OpenAPI attachment is not available, so the key check, name and limit codes are named here; the browser contract confirms or renames them.

MemberSiteIdentityPort, HpcEligibilityPort and AupPort supply the gates; answers that do not match the requested member and Site raise InputsUnavailable for retry. Clock and IdGenerator supply time and identities (adaptors.system). SshAccessSettings holds the Site, propagation window, key limit and login nodes.

#Persistence

Private database pharos_ssh_access with a restricted runtime role. One pharos.aggregates row of kind member_ssh_access holds the root snapshot. The same transaction writes the command receipt and:

• pharos.ssh_access_keys: one row per key, unique per root and fingerprint;
• pharos.ssh_access_work: whether the root has unacknowledged generations or present keys, and when its earliest propagation window started;
• pharos.ssh_access_site_results: one immutable row per recorded (key_id, generation).

A database guard rejects child writes outside their root's command transaction or at another root version. The adaptor checks every row against the locked snapshot before a decision; a mismatch is CorruptState. Each change also records the browser projection's realtime publication; the aggregate publishes no integration events.

#Transport and composition

adaptors.messaging registers the API's SSH access requests with the shared RequestRegistry in metallo.adaptors.requests. Commands answer with their recorded outcome, whose reply commits in the same transaction. ActorMembers maps the authenticated actor to a member; an unmapped actor gets member_not_found. metallo.services.ssh_access composes the request consumers, reply and realtime relays, propagation, owner projection and policy workers in the ssh-access Deployment. The normal composition uses the owner adaptors below. adaptors.fixture_owners remains available when a development test explicitly sets SSH_ACCESS_FIXTURE_FILE; it refuses to load outside development configuration. Installed manifests use canonical owners, enforced by deployment checks.

#Owner inputs

adaptors.owner_views.SshAccessView keeps this context's own Party, provider binding, Project, membership and Project authority tables. Migrations 0002 and 0003 add these beside the existing key aggregate. Eight durable Beto subscriptions use SSH_ACCESS_VIEWS_BROKER_URL, the pharos_ssh_access_views identity. Consumers start before checkpoint synchronisation; snapshots repair missed events and the view becomes ready only after matching Beto's checkpoints. Repair includes Parties referenced by saved key roots after they leave Projects.

adaptors.canonical_owners interprets the API actor as the canonical Party ID. A fresh Beto Party snapshot repairs authentication and binding gaps on member requests; a newer event already saved takes precedence. Site results publish to that same Party principal. The Site agent needs only Metallo's Site API and LDAP.

Both supported Beto membership roles hold project.view. A Project needs a matching ready or changing allocation; other offerings, Sites, pending, suspended and retired resources grant no new key access. Resource evidence expires after fifteen minutes. The numeric UID comes from the existing HPC offering-user reconciliation, which checks Waldur's username against the canonical Site binding. Resource Provisioning stores it separately from the LDAP entry UUID; the numeric value is required explicitly, including after an upgrade. SSH Access requires the exact Party and binding, a ready account and a positive POSIX UID; conflicting reports defer the operation. The report timestamp supplies the identity evidence revision. These are Resource Provisioning owner queries, composed with its own database credentials.

adaptors.legacy_policy queries accepted and required versions of the retained platform AUP. Waldur Terms of Service are a separate policy. Acceptance replies must name the requested Party and its current GitLab binding and carry recent evidence. TLS verifies the private authority; redirects and inherited proxies are refused. An unavailable or malformed answer, including a broken HTTP response, defers new submissions and resubmissions.

A confirmed missing Party or current GitLab binding means no current acceptance can be established. The reader then fetches the required version through the binding-independent private policy GET, returning no accepted version. The same rule applies when the policy POST confirms that the exact requested binding is no longer current. Failed owner queries remain unavailable inputs; they never become evidence of absence or a fabricated policy version.

The policy worker checks roots holding present keys every minute. When acceptance differs from the current version, it issues the existing deterministic revocation command per root, AUP version and observed key generations. A later resubmission can therefore be revoked if confirmed binding loss means current acceptance can no longer be proved, even when the required version remains unchanged. A delayed command affects only its observed generations. The durable work index supports retry after an outage or restart; one unavailable member does not block the rest. Existing keys remain discoverable and removable when Project membership or account evidence disappears. Membership loss alone does not revoke existing keys.

The suspension and reinstatement policy is defined in DD-0055 §2.2. AccessRevocation compares each present key's grant with current Party authority and security-retired Site bindings. A changed authentication generation withdraws the old grant even when suspension and reinstatement both happened during an interruption. Reinstatement leaves the old key revoked until the member explicitly resubmits after normal gate checks. Retirement never changes the stored account target: removal still addresses the original LDAP entry.

Committed Party and binding events wake the access worker. Its one-minute repair sweep uses the existing durable key index; unavailable owner queries defer that member without blocking others. This worker does not query AUP, so a policy outage cannot prevent a confirmed security withdrawal. Generation-specific commands and Site acknowledgements protect newer key submissions from delayed work. The API still requires an authenticated Party for member commands; automatic withdrawal handles disabled members who cannot use that route.

UID and AUP events can replace these query adaptors when their authoritative contexts publish suitable contracts, including initial-load and repair queries. The gates and Site protocol retain their current ports.

#Browser projection

adaptors.projection encodes pharos.realtime.v1.MemberSshAccessProjection: the member, the Site and the visible keys with the #678 fields except the key value. The PostgreSQL boundary records it with every committed root revision, including Site results and propagation completion. ActorMembers supplies one principal-targeted publication per actor; a member with no actor has no browser audience. The projection relay sends each exact persisted envelope through the restricted realtime gateway. The browser reads the owner's ProjectionSnapshot, applies only newer revisions per root and reconciles once after unrecoverable history.

#Site protocol

adaptors.site_http serves POST /v1/site/ssh-access/pull, snapshot and acknowledge from the Site contract on the Metallo Site API listener. adaptors.native_callers accepts only native bearers whose grant has the ssh-access scope; the Site comes from that grant, and an unconfigured Site is refused. Pull and snapshot pages use route- and Site-bound page tokens; memberRef is the root identity. Bodies are strict JSON, duplicate fields and nulls are refused, and every reply is checked against the generated contract models before it is sent. Errors are redacted: 401 credentials, 403 scope or Site, 409 conflicting results, 422 invalid input or member references, 503 unavailable credentials or storage.

#Access agent

application.agent.AccessAgent converges one member at a time on the Site host. It finds exactly one entry with uid=<username> under the people base; a missing entry leaves the work pending, because Waldur marks the account OK before the Site agent creates it. A different uidNumber blocks every write and raises an operational alert. For each key it adds the exact line or deletes the values with that fingerprint, reads the entry back and acknowledges only what the read-back proves. Already present and already absent count as success. A permanent refusal of an add is acknowledged as rejected with the directory's reason; a refused removal stays pending with an alert.

The pull pass acknowledges every converged key, which recovers a crash between an LDAP change and its acknowledgement. The snapshot pass repairs drift, acknowledges only keys it changed and alerts on sshPublicKey values Pharos does not hold. A removal after a recorded rejection replays, since nothing remains.

adaptors.ldap_directory implements only search, value add, value delete and read-back with ldap3. Every modification carries a critical RFC 4528 assertion for the entry's uidNumber, immutable entryUUID and freshly read entryCSN. The first key and ldapPublicKey class are added in one modify; the last key and class are removed in one modify. Every value change reads again, including successive removals from a stale caller entry. Contention retries from fresh state; result 20 alone is not proof that a compound modify stored a key. This supports both MUST and MAY schemas. The bind may write only that class value, not other classes or POSIX attributes. Read-back verifies the same identity, so a replacement account at the same DN cannot receive the old key. It uses SASL EXTERNAL over a local ldapi socket without a password, or simple bind over verified LDAPS; cleartext LDAP is restricted to loopback. adaptors.site_client calls the Site routes over verified HTTPS without proxies or redirects and validates every reply against the contract. metallo.services.ssh_access_agent composes them; packaging and directory ACLs are in deploy/ssh-access-agent.

#Status

The context, browser/API contracts, Site protocol, access agent, Beto projections, resource queries and retained AUP reader/revocation worker are implemented. An installation must qualify its own AAI Site binding, offering, policy reader and real member journey before activation; see the deployment guide. Fixture qualification and live deployment qualification are separate evidence.

#Verification

• Unit: services/metallo/tests/ssh_access covers the #678 transition table, key validation, gate order, generation fencing, the propagation window, deleted keys awaiting removal, UID changes, policy transport failures and validate/submit parity. Resource Provisioning's test_hpc_access.py covers offering scope, freshness, account correlation and conflicting reports.
• PostgreSQL: services/metallo/tests/infrastructure/ssh_access.py, run by pnpm test:integration, covers concurrent submissions, duplicates in every state, lost-response replay, resubmission, stale and replayed Site results, AUP revocation, rollback, corrupt indexes and the restricted role. The same lane drives the HTTP routes through the API, RabbitMQ and the ssh-access image.
• Beto/Metallo: the cross-service integration group runs tests/infrastructure/ssh_access_owner.py with real Beto, PostgreSQL and broker identities. It checks event delivery, checkpoint rebuild, pending authority, membership retirement and canonical Site bindings. A labelled policy HTTP fixture and directory double exercise submission and AUP removal through the production Site routes and agent codecs.
• API contract: services/api/adaptors/http/ssh_access_test.go checks every route and error code against the bundled OpenAPI document, the gated 403, validate/submit parity, origin, content type, command identity and preservation of the owner's projection bytes.
• Browser projection: tests/ssh_access/test_projection.py covers snapshot bytes, deleted keys and per-actor audiences; e2e/ssh-access-realtime.spec.ts covers first read, recovered history and one reconciliation after history loss.
• Site API: tests/ssh_access/test_site_http.py and tests/test_site_credentials.py cover grant separation in both directions, expired, revoked and unknown bearers, the Site taken from the grant, strict bodies and page tokens. The integration lane pulls and acknowledges through the deployed site-api image with issued grants.
• Access agent: tests/ssh_access/test_agent.py with directory and Site doubles; scripts/ssh_access_ldap_integration.py (also run by pnpm test:integration) runs the agent against OpenLDAP with openssh-lpk and both ACLs: ACL enforcement in both directions, missing entry, uidNumber mismatch, unmanaged values, a crash after modify and drift repair.

The focused uv run --project devops --frozen python scripts/ssh_access_must_integration.py lane uses disposable osixia/openldap:1.5.0, replaces its openssh-lpk class with MUST sshPublicKey and reads the schema back before testing. It applies the actual ACL template and checks first/last key changes, stale entries, concurrent writers, and result 50 for other class values and attributes. The container is removed on exit.

Page source

Source snapshot · Revision f928834c8.