# Source Control Source Control owns repository/provider bindings and delegated source-control effects for canonical Beto Projects. It reconciles one private GitLab group per Project and its direct members, and it resolves Project repositories and reads their action files at exact commits. It also owns recoverable, repository-scoped credentials for accepted Run operations. ## Ownership and invariants - Beto Party Governance owns `PartyId`, current authentication eligibility and verified external-account associations. - Beto Project owns `ProjectId`, lifecycle, membership, roles and access policy. - Source Control owns its Beto projections, durable GitLab work, GitLab group bindings and member effects. GitLab IDs and paths are provider bindings. - Source Control owns canonical repository IDs. Each is bound to one opaque provider identity `(provider, instance, external_id)` and survives renames and transfers; a deleted repository stays recorded as missing. - Account creation is a separate workflow. The retained compatibility account integration is its sole writer during coexistence. Source Control never creates a GitLab user, searches by email or derives ownership from a username. - A member needs current Party eligibility and exactly one established binding for the configured GitLab instance. The binding's opaque external value must be a numeric GitLab user ID. Missing, ambiguous or unverified prerequisites stay pending. `site_username` is optional owner input for the separate account writer and is not an input to membership reconciliation. - Installation administration creates the configured root group. The runtime token cannot create top-level groups, and the root never receives Project members. Project groups are retained on closure; closure removes their direct managed memberships. The context uses dedicated PostgreSQL credentials. Its persisted aggregate kinds include `project_access`, `party_account`, private `run_repository_grant` records and `repository_publication` consent; the original draft snapshot identities are preserved; its migration adds the `source_repositories` identity table. Restoring the draft `party_account` and `project_access` JSON shapes is supported; restored accounts are deliberately unverified until Party Governance confirms them. The draft database and workload were never activated, so the context/database owner changed from `gitlab_access` to `source_control` before supported operational data existed. ## Inputs and durable delivery The dedicated Beto caller `pharos_source_control_beto` owns seven queues: - `PartyRegistered` starts a pending authentication prerequisite; - `PartyAuthenticationChanged` updates current eligibility; - `PartyProviderBindingEstablished` and `PartyProviderBindingRetired` update the configured GitLab association; - `ProjectChanged` updates complete material Project state; - `ProjectAccessChanged` has a separate receipt and forces an owner repair. It grants no lasting permission itself; - `ProjectAuthorityChanged` immediately fences older authority replies and requests the latest complete owner snapshot. All payloads use the shared version 1 Protobuf envelope and generated Party/Project messages. Decoders reject unknown fields, owner/visibility mismatches and payload identities that disagree with the envelope. Project member provider IDs are validated at the transport boundary and discarded; the projection retains only canonical `PartyId`, roles and `ProjectId`. Beto provisions the cross-service queues, retry/dead-letter paths and grants. Source Control commits the receipt, projection change and pending reconciliation work before the delivery can be acknowledged. A GitLab failure records failed work and causes broker retry; the periodic pass retries saved work after restart and repairs manual GitLab drift. ## Authority freshness and revocation `ProjectChanged` is material state and carries no freshness assertion. A scoped `GetProjectIntegrationSnapshot` reply supplies separate source state, evidence and `validUntil`. New memberships and role increases require `ready`, unexpired evidence for the exact retained Project revision. When authority is pending, failed, stale or expired, reconciliation retains only access that was already applied under confirmed authority, capped at the previous level. Current removals and role reductions still take effect. Party disablement, binding retirement and Project closure remove access immediately. A delayed binding or GitLab account cannot turn stale Project evidence into a new grant. When the owner answers `not_found` for a Project, it no longer exposes it: a definite answer, not a failure to retry. A Project never recorded here is settled without a write. A recorded one becomes `absent` and, like a closed Project, keeps no one in its group; a later complete owner snapshot brings it back. Only an unavailable owner or a GitLab failure still asks the delivery for a retry. A Project records how many times it has been withdrawn. Withdrawal, owner confirmation and reconciliation commands include that count, so a withdrawal followed by a restoration with identical evidence never replays an earlier cycle's receipts, while a retry within one cycle keeps its identity. The Project is read before the owner is asked. A `not_found` reply withdraws it only if its revision, authority, source state, evidence and count are unchanged, and a positive reply confirms it only if no withdrawal was recorded meanwhile. If the saved Project moved, by newer authority, a withdrawal or local expiry of its evidence, the reply may describe an earlier moment: it is neither applied nor given a receipt, and the owner is asked again, up to three times before the delivery is retried. Settling such a reply as stale would lose a withdrawal for good, because the owner inventory no longer lists that Project. A replayed receipt returns before its decision runs, so after every owner answer the saved Project is checked again: absent after a `not_found`, and with the observed withdrawal count after a positive reply. Otherwise the owner is asked again. "Already applied" means what the last reconciliation really left in GitLab for each Party: its account and its level. A decision whose GitLab update failed is not applied, so a later run cannot apply it once its authority has expired. A step that GitLab refuses does not stop the others: the run applies every step it can, removals and downgrades included, keeps what GitLab accepted, and reports the first refusal for a retry. What it merely observes in GitLab never raises the saved level: only a step that succeeded, or that was not needed, does, so a refused corrective downgrade leaves the cap where it was. Both supported Beto roles, `developer` and `project-administrator`, map to Maintainer of the Project's GitLab group under the temporary #1001 policy. Members can create repositories in the kind groups and push to the default branch under the installed Maintainer-only protection. Unknown roles grant no access. `MEMBER_LEVEL` in `domain/access.py` controls the developer mapping. Existing groups gain this level through periodic repair once the owner supplies fresh authority; expired authority retains the previously applied ceiling. Waldur roles and Beto's Project membership administration permissions remain unchanged. Run tokens retain Developer access. The [member-permission qualification](../../../../docs/qualification/project-member-permissions-1001.md) records creation, protected-branch push and revocation against the deployed owners. Source Control creates each kind group with `project_creation_level` Maintainer, and the Project group with no one allowed to create a repository in it; GitLab does not inherit the setting from the root group or from the Project group. Each Project projection records `pending`, `applying`, `applied` or `failed` work, attempt count, delayed/conflicting Parties, unresolved GitLab users and redacted failure class. `/diagnostics` exposes counts only; it contains no canonical or provider identifiers. ## Loading and repair The application exposes owner ports for: 1. an explicitly paginated canonical Project inventory; 2. a complete Project integration snapshot with source evidence; 3. current Party authentication; and 4. singular provider-binding repair through the published Party contract. Composition starts durable consumers before reading the Project inventory. Each page is repaired through the same revision checks as live delivery, so changes during loading, duplicate/reordered events and a repeated bootstrap converge. Re-running the inventory rebuilds a lost Source Control projection. A process lock serialises each Project's owner lookup, answer application and reconciliation inside this single-replica slice, including before its first record exists. An absence reply therefore cannot overtake a concurrent first confirmation. Projects that remain unknown still need no receipt or tombstone. The lock is reentrant so answer application can reconcile the same Project; other Projects proceed independently. The production adaptor uses Beto’s scoped checkpoint inventory and singular Project snapshots, plus complete Party snapshots including retired bindings. It verifies the inventory manifest, preserves absolute authority expiry and orders authority independently of material Project revisions. The periodic repair repeats that inventory and the Party snapshots; existing bindings therefore recover even when no new creation event is emitted. `PROJECT_SOURCE_SCOPE` selects the exact scope enrolled for this caller. Deployment remains paused pending #771 qualification. ## GitLab adaptor and process `metallo.services.source_control:create_app` composes the seven consumers, initial load and periodic repair with one Source Control database. The GitLab adaptor has a closed operation set: find/create/rename the configured child group, list its subgroups, list direct members, resolve a user by numeric ID, set Developer/Maintainer and remove a direct member. Calls use HTTPS, the configured hostname and mounted CA. Runtime mounts only: - `SOURCE_CONTROL_DATABASE_URL`; - `SOURCE_CONTROL_BETO_BROKER_URL` and its enrolled `PROJECT_SOURCE_SCOPE`; - a scoped `SOURCE_CONTROL_GITLAB_TOKEN`; and - the public `SOURCE_CONTROL_GITLAB_CA_CERT`. Installation and recovery administrator credentials stay in administrative tooling. Network policy admits PostgreSQL, RabbitMQ, DNS and HTTPS for the trusted GitLab endpoint. The enabling overlay must set the URL, instance and pre-provisioned root group and provide a token limited to the operations above. On a site, the `distributed` overlay starts it, the root adds the GitLab URL from the edge's GitLab origin and the instance from the API authentication binding, and delivery makes the root group, the service users and their tokens (`platform_deploy/source_control.py`). A site's GitLab is publicly trusted, so the workloads read the image's CA bundle and no CA key is written. ## Repositories and action files `application.repositories.Repositories` resolves a repository by its `{kind}/{Project ID}/{name}` path or clone URL, answers the Project a namespace names, reads `actions.yaml` at the commit a branch, tag or commit names, and describes a known repository's last state. Access is the rule that grants provider membership above, and hidden, moved and absent repositories all answer `None`. The domain and application name no provider; `adaptors.gitlab_repositories` maps them onto GitLab with a separate read-only token (`SOURCE_CONTROL_GITLAB_READ_TOKEN_FILE`). `adaptors.repositories.repository_reads` composes the queries for other contexts, and `metallo.adaptors.source_control_actions` implements Workload Portfolio's `RepositoryActions` over them, and `metallo.adaptors.source_control_owners` composes them for `metallo-requests` (both `RepositoryDirectory` ports and `RepositoryActions`) and the execution worker (`SourceVersions`). Those two processes mount `SOURCE_CONTROL_DATABASE_URL`, the read-only `SOURCE_CONTROL_GITLAB_READ_TOKEN` and the CA, never the managing token, and refuse to start without the GitLab connection values (#796). [Design and verification](../../docs/epics/project-provisioning/source-control-repositories.md). ## Follow-up boundaries The repository queries are connected to the request host and the execution worker (#796); the overlay that connects a GitLab sets their URL, instance and root group as for this process, and provisions the read token. Repository changes are not yet published as events (#797). The installed worker stays paused until the real-owner Run journey is qualified. Execution Core receives no GitLab credential; Site Runtime receives only the accepted operation's repository credential over its authenticated HTTPS connection. ## Run repository credentials The Site API joins Execution Core's immutable Run/Attempt material, current Project authority and Resource Provisioning's applied execution identity. Source Control then refreshes canonical repository ownership with GitLab. A request must match a saved repository, logical input, branch and exact commit. The initiating member's later removal does not revoke accepted work. Project closure, terminal/cancelled work, a changed binding or an expired original dispatch deadline prevents issuance. Each Attempt/input/material phase has a private grant aggregate. Issuance records pending intent before calling GitLab and seals the returned token before delivery. A lost create response is recovered by the stable provider name: any surviving token is revoked before retry. Revocation records its intent first and is retried after restart. A 15-second recovery pass retires grants whose owning work has ended, even if the Runtime disappears. Short fetches can release a token and stage-out can obtain a new generation later while the original work still permits it. Provider failures remain pending; the GitLab expiry bounds credentials while revocation is unavailable. GitLab project access tokens use Developer access with `write_repository` for one repository, covering Git HTTPS and LFS. Branch selection describes the checkout and result destination; the token itself is repository-scoped. GitLab's protected-branch rules enforce write restrictions and must be qualified against the installed provider. Native expiry is the next UTC midnight, which must fit inside the accepted deadline. A midnight closer than `MINIMUM_LIFETIME` (15 minutes) is skipped for the one after it when the deadline allows that, so a credential is never handed out with less than that left; an existing credential with less left is replaced. Near the deadline, or for output recovery, whose deadline is one day from the request, the nearer midnight stays and the token is short. A provider failure reaches the caller as `TokenProviderUnavailable` (no answer, 408, 429 or 5xx: a later attempt may succeed) or as `TokenProviderRefused` (any other answer, or an issuer that is not an external non-administrator: the same request is refused again until the installation changes, for example when the issuer has lost its Maintainer access). Both carry the HTTP status and never the response body, and both are `TokenProviderError`. The issuer is a dedicated **external**, non-administrator GitLab service account with a personal access token and sufficient access to the managed Project repositories. An internal issuer would give its project tokens broader internal-repository visibility. `SOURCE_CONTROL_RUN_ISSUER_TOKEN_FILE` and `SOURCE_CONTROL_RUN_GRANT_SEALING_KEY_FILE` are mounted only in the owner composition; the latter holds 32 random bytes encoded as hex or base64. Recovery depends on retaining that sealing key. Plaintext credentials never enter broker events, projections, receipts or operator logs. Revocation needs the original Site and saved identity, and remains available after personal access or Project execution authority has ended. ## Member repository management The Source Control process answers typed `source_control` query/command requests from the API using its own `SOURCE_CONTROL_REQUESTS_BROKER_URL`. It resolves retained paths to canonical repository IDs and reads repository metadata and deletion status. Beto’s current `project.view` gates every read; the dynamic `can_delete` flag comes from `repositories.delete`, reserved for Project administrators. Provider IDs remain private bindings. A `repository_deletion` aggregate is keyed by canonical repository ID. It stores the requesting Party, Project and exact provider binding. `DeleteRepository` requires a command ID and expected deletion version; its snapshot, receipt and command reply commit together. Authority checks and GitLab calls stay outside that transaction. A recovery worker scans complete pending work, rechecks current Beto authority and the saved native repository’s Project, then uses the existing scoped GitLab management credential. Repeated calls address the same native ID. A transfer or revoked access records `refused`; transient failures leave `pending`. A new command at the observed version can restart a refused request. Acceptance records `scheduled`, including GitLab’s delayed deletion. Repository presence remains the last provider observation; acceptance does not assert physical removal. The process drains request, reply and recovery workers before closing its database. HTTP contracts live in `contracts/source_control/http_api/`; the public surface is in the API service. Members keep Developer/Maintainer GitLab roles. No GitLab administrator credential enters this process. Repository creation, catalogue, content and fork commands still use the retained compatibility routes. ## Public repository publication `application.publications.Publications` owns publication consent for one canonical repository. The authoring repository remains inside its private Project group. An authorised `SetRepositoryPublication` command records the Party, canonical Project and repository, original provider binding, desired visibility and expected publication version. Its receipt and command reply commit with the aggregate. Current `project.manage` authority is required for publication and withdrawal; repository reads expose `can_publish`, desired visibility, state and version. The states are `pending`, `published`, `private` and `refused`. Provider I/O runs after the consent transaction. The worker first reserves a private provider copy and records its opaque binding before copying content or enabling public access. A stable reserved topic identifies the intended copy if a create response is lost. A conflicting topic, provider identity, Project path or shared-group relationship prevents adoption. Canonical repository identity continues to name the authoring repository; the public copy is a binding of the publication aggregate. Installation creates the separate public `pharos-public` root. Only the existing Source Control manager and the installation administrator may be direct members of that root. Unexpected membership, sharing or a private namespace at that path stops installation. Publication creates public subgroups using canonical Project IDs and repository kinds, with copies at `pharos-public///`. Project display names, group shares and Project memberships are never copied into the public hierarchy. The authoring namespace and Run credential issuer keep their existing scope. The Git adaptor copies heads, tags and Git LFS objects into a temporary bare repository. It uses verified HTTPS URLs, the mounted CA and the scoped manager credential. Credentials travel only in the child environment. Imported working trees, hooks, submodules, custom LFS transfers and credential helpers do not run. LFS endpoints are fixed to the two approved repositories. Complete LFS upload precedes the atomic ref push. A changed default branch is installed before obsolete refs are pruned, and public visibility is enabled last. Branch protections are removed only from the verified managed copy so it can follow source history rewrites; the private author's protections remain in force. The public copy has pipelines, runners and unrelated project features disabled. Current authority is rechecked before sending new content and before enabling public visibility. Revoked authority, a missing source or a source moved to another Project withdraws the owned public copy and records `refused`. A new authorised command can retry. Explicit withdrawal sets only the owned copy private and remains durable during a provider outage. A removed public copy is reserved again before publication. Renames and moves retain the public provider identity. After a Project transfer, the new Project can still read canonical repository metadata. Its publication status stays private and pending until the old publication is withdrawn. It can then submit new consent, which rehomes the same private copy before publishing under the new Project's path. The saved previous copy Project and provider binding recover a lost transfer response. Pending work is retried every five seconds. Published repositories are checked every thirty seconds; equal Git refs avoid a full copy. A published state records the last completed copy, so subsequent authoring edits can take one repair interval to appear. Provider failures retain durable work for retry. These server workers introduce no browser polling; the UI can request the current status on an explicit refresh. The workload bounds temporary storage to 2 GiB, and each Git command to five minutes. Shutdown interrupts the owned Git process group and stops subsequent provider work before the database closes. Source Control requires `git` and `git-lfs` in its image, plus the existing GitLab HTTPS network access.