For AI agents: the complete documentation index is available at /llms.txt, the full documentation bundle is available at /llms-full.txt, and this page is available as Markdown at /reference/services/metallo/contexts/source_control/README.md.

Source Control

Source Control owns repository/provider bindings and delegated source-control effects for canonical Beto Projects. It reconciles one private GitLab group per Project and its direct members, and it resolves Project repositories and reads their action files at exact commits. It also owns recoverable, repository-scoped credentials for accepted Run operations.

Ownership and invariants

  • Beto Party Governance owns PartyId, current authentication eligibility and verified external-account associations.
  • Beto Project owns ProjectId, lifecycle, membership, roles and access policy.
  • Source Control owns its Beto projections, durable GitLab work, GitLab group bindings and member effects. GitLab IDs and paths are provider bindings.
  • Source Control owns canonical repository IDs. Each is bound to one opaque provider identity (provider, instance, external_id) and survives renames and transfers; a deleted repository stays recorded as missing.
  • Account creation is a separate workflow. The retained compatibility account integration is its sole writer during coexistence. Source Control never creates a GitLab user, searches by email or derives ownership from a username.
  • A member needs current Party eligibility and exactly one established binding for the configured GitLab instance. The binding's opaque external value must be a numeric GitLab user ID. Missing, ambiguous or unverified prerequisites stay pending. site_username is optional owner input for the separate account writer and is not an input to membership reconciliation.
  • Installation administration creates the configured root group. The runtime token cannot create top-level groups, and the root never receives Project members. Project groups are retained on closure; closure removes their direct managed memberships.

The context uses dedicated PostgreSQL credentials. Its persisted aggregate kinds include project_access, party_account, private run_repository_grant records and repository_publication consent; the original draft snapshot identities are preserved; its migration adds the source_repositories identity table. Restoring the draft party_account and project_access JSON shapes is supported; restored accounts are deliberately unverified until Party Governance confirms them. The draft database and workload were never activated, so the context/database owner changed from gitlab_access to source_control before supported operational data existed.

Inputs and durable delivery

The dedicated Beto caller pharos_source_control_beto owns seven queues:

  • PartyRegistered starts a pending authentication prerequisite;
  • PartyAuthenticationChanged updates current eligibility;
  • PartyProviderBindingEstablished and PartyProviderBindingRetired update the configured GitLab association;
  • ProjectChanged updates complete material Project state;
  • ProjectAccessChanged has a separate receipt and forces an owner repair. It grants no lasting permission itself;
  • ProjectAuthorityChanged immediately fences older authority replies and requests the latest complete owner snapshot.

All payloads use the shared version 1 Protobuf envelope and generated Party/Project messages. Decoders reject unknown fields, owner/visibility mismatches and payload identities that disagree with the envelope. Project member provider IDs are validated at the transport boundary and discarded; the projection retains only canonical PartyId, roles and ProjectId.

Beto provisions the cross-service queues, retry/dead-letter paths and grants. Source Control commits the receipt, projection change and pending reconciliation work before the delivery can be acknowledged. A GitLab failure records failed work and causes broker retry; the periodic pass retries saved work after restart and repairs manual GitLab drift.

Authority freshness and revocation

ProjectChanged is material state and carries no freshness assertion. A scoped GetProjectIntegrationSnapshot reply supplies separate source state, evidence and validUntil. New memberships and role increases require ready, unexpired evidence for the exact retained Project revision.

When authority is pending, failed, stale or expired, reconciliation retains only access that was already applied under confirmed authority, capped at the previous level. Current removals and role reductions still take effect. Party disablement, binding retirement and Project closure remove access immediately. A delayed binding or GitLab account cannot turn stale Project evidence into a new grant.

When the owner answers not_found for a Project, it no longer exposes it: a definite answer, not a failure to retry. A Project never recorded here is settled without a write. A recorded one becomes absent and, like a closed Project, keeps no one in its group; a later complete owner snapshot brings it back. Only an unavailable owner or a GitLab failure still asks the delivery for a retry.

A Project records how many times it has been withdrawn. Withdrawal, owner confirmation and reconciliation commands include that count, so a withdrawal followed by a restoration with identical evidence never replays an earlier cycle's receipts, while a retry within one cycle keeps its identity. The Project is read before the owner is asked. A not_found reply withdraws it only if its revision, authority, source state, evidence and count are unchanged, and a positive reply confirms it only if no withdrawal was recorded meanwhile. If the saved Project moved, by newer authority, a withdrawal or local expiry of its evidence, the reply may describe an earlier moment: it is neither applied nor given a receipt, and the owner is asked again, up to three times before the delivery is retried. Settling such a reply as stale would lose a withdrawal for good, because the owner inventory no longer lists that Project. A replayed receipt returns before its decision runs, so after every owner answer the saved Project is checked again: absent after a not_found, and with the observed withdrawal count after a positive reply. Otherwise the owner is asked again.

"Already applied" means what the last reconciliation really left in GitLab for each Party: its account and its level. A decision whose GitLab update failed is not applied, so a later run cannot apply it once its authority has expired. A step that GitLab refuses does not stop the others: the run applies every step it can, removals and downgrades included, keeps what GitLab accepted, and reports the first refusal for a retry. What it merely observes in GitLab never raises the saved level: only a step that succeeded, or that was not needed, does, so a refused corrective downgrade leaves the cap where it was.

Both supported Beto roles, developer and project-administrator, map to Maintainer of the Project's GitLab group under the temporary #1001 policy. Members can create repositories in the kind groups and push to the default branch under the installed Maintainer-only protection. Unknown roles grant no access. MEMBER_LEVEL in domain/access.py controls the developer mapping. Existing groups gain this level through periodic repair once the owner supplies fresh authority; expired authority retains the previously applied ceiling. Waldur roles and Beto's Project membership administration permissions remain unchanged. Run tokens retain Developer access. The member-permission qualification records creation, protected-branch push and revocation against the deployed owners.

Source Control creates each kind group with project_creation_level Maintainer, and the Project group with no one allowed to create a repository in it; GitLab does not inherit the setting from the root group or from the Project group.

Each Project projection records pending, applying, applied or failed work, attempt count, delayed/conflicting Parties, unresolved GitLab users and redacted failure class. /diagnostics exposes counts only; it contains no canonical or provider identifiers.

Loading and repair

The application exposes owner ports for:

  1. an explicitly paginated canonical Project inventory;
  2. a complete Project integration snapshot with source evidence;
  3. current Party authentication; and
  4. singular provider-binding repair through the published Party contract.

Composition starts durable consumers before reading the Project inventory. Each page is repaired through the same revision checks as live delivery, so changes during loading, duplicate/reordered events and a repeated bootstrap converge. Re-running the inventory rebuilds a lost Source Control projection. A process lock serialises each Project's owner lookup, answer application and reconciliation inside this single-replica slice, including before its first record exists. An absence reply therefore cannot overtake a concurrent first confirmation. Projects that remain unknown still need no receipt or tombstone. The lock is reentrant so answer application can reconcile the same Project; other Projects proceed independently.

The production adaptor uses Beto’s scoped checkpoint inventory and singular Project snapshots, plus complete Party snapshots including retired bindings. It verifies the inventory manifest, preserves absolute authority expiry and orders authority independently of material Project revisions. The periodic repair repeats that inventory and the Party snapshots; existing bindings therefore recover even when no new creation event is emitted. PROJECT_SOURCE_SCOPE selects the exact scope enrolled for this caller. Deployment remains paused pending #771 qualification.

GitLab adaptor and process

metallo.services.source_control:create_app composes the seven consumers, initial load and periodic repair with one Source Control database. The GitLab adaptor has a closed operation set: find/create/rename the configured child group, list its subgroups, list direct members, resolve a user by numeric ID, set Developer/Maintainer and remove a direct member. Calls use HTTPS, the configured hostname and mounted CA.

Runtime mounts only:

  • SOURCE_CONTROL_DATABASE_URL;
  • SOURCE_CONTROL_BETO_BROKER_URL and its enrolled PROJECT_SOURCE_SCOPE;
  • a scoped SOURCE_CONTROL_GITLAB_TOKEN; and
  • the public SOURCE_CONTROL_GITLAB_CA_CERT.

Installation and recovery administrator credentials stay in administrative tooling. Network policy admits PostgreSQL, RabbitMQ, DNS and HTTPS for the trusted GitLab endpoint. The enabling overlay must set the URL, instance and pre-provisioned root group and provide a token limited to the operations above. On a site, the distributed overlay starts it, the root adds the GitLab URL from the edge's GitLab origin and the instance from the API authentication binding, and delivery makes the root group, the service users and their tokens (platform_deploy/source_control.py). A site's GitLab is publicly trusted, so the workloads read the image's CA bundle and no CA key is written.

Repositories and action files

application.repositories.Repositories resolves a repository by its {kind}/{Project ID}/{name} path or clone URL, answers the Project a namespace names, reads actions.yaml at the commit a branch, tag or commit names, and describes a known repository's last state. Access is the rule that grants provider membership above, and hidden, moved and absent repositories all answer None. The domain and application name no provider; adaptors.gitlab_repositories maps them onto GitLab with a separate read-only token (SOURCE_CONTROL_GITLAB_READ_TOKEN_FILE). adaptors.repositories.repository_reads composes the queries for other contexts, and metallo.adaptors.source_control_actions implements Workload Portfolio's RepositoryActions over them, and metallo.adaptors.source_control_owners composes them for metallo-requests (both RepositoryDirectory ports and RepositoryActions) and the execution worker (SourceVersions). Those two processes mount SOURCE_CONTROL_DATABASE_URL, the read-only SOURCE_CONTROL_GITLAB_READ_TOKEN and the CA, never the managing token, and refuse to start without the GitLab connection values (#796). Design and verification.

Follow-up boundaries

The repository queries are connected to the request host and the execution worker (#796); the overlay that connects a GitLab sets their URL, instance and root group as for this process, and provisions the read token. Repository changes are not yet published as events (#797). The installed worker stays paused until the real-owner Run journey is qualified. Execution Core receives no GitLab credential; Site Runtime receives only the accepted operation's repository credential over its authenticated HTTPS connection.

Run repository credentials

The Site API joins Execution Core's immutable Run/Attempt material, current Project authority and Resource Provisioning's applied execution identity. Source Control then refreshes canonical repository ownership with GitLab. A request must match a saved repository, logical input, branch and exact commit. The initiating member's later removal does not revoke accepted work. Project closure, terminal/cancelled work, a changed binding or an expired original dispatch deadline prevents issuance.

Each Attempt/input/material phase has a private grant aggregate. Issuance records pending intent before calling GitLab and seals the returned token before delivery. A lost create response is recovered by the stable provider name: any surviving token is revoked before retry. Revocation records its intent first and is retried after restart. A 15-second recovery pass retires grants whose owning work has ended, even if the Runtime disappears. Short fetches can release a token and stage-out can obtain a new generation later while the original work still permits it. Provider failures remain pending; the GitLab expiry bounds credentials while revocation is unavailable.

GitLab project access tokens use Developer access with write_repository for one repository, covering Git HTTPS and LFS. Branch selection describes the checkout and result destination; the token itself is repository-scoped. GitLab's protected-branch rules enforce write restrictions and must be qualified against the installed provider. Native expiry is the next UTC midnight, which must fit inside the accepted deadline. A midnight closer than MINIMUM_LIFETIME (15 minutes) is skipped for the one after it when the deadline allows that, so a credential is never handed out with less than that left; an existing credential with less left is replaced. Near the deadline, or for output recovery, whose deadline is one day from the request, the nearer midnight stays and the token is short.

A provider failure reaches the caller as TokenProviderUnavailable (no answer, 408, 429 or 5xx: a later attempt may succeed) or as TokenProviderRefused (any other answer, or an issuer that is not an external non-administrator: the same request is refused again until the installation changes, for example when the issuer has lost its Maintainer access). Both carry the HTTP status and never the response body, and both are TokenProviderError.

The issuer is a dedicated external, non-administrator GitLab service account with a personal access token and sufficient access to the managed Project repositories. An internal issuer would give its project tokens broader internal-repository visibility. SOURCE_CONTROL_RUN_ISSUER_TOKEN_FILE and SOURCE_CONTROL_RUN_GRANT_SEALING_KEY_FILE are mounted only in the owner composition; the latter holds 32 random bytes encoded as hex or base64. Recovery depends on retaining that sealing key. Plaintext credentials never enter broker events, projections, receipts or operator logs. Revocation needs the original Site and saved identity, and remains available after personal access or Project execution authority has ended.

Member repository management

The Source Control process answers typed source_control query/command requests from the API using its own SOURCE_CONTROL_REQUESTS_BROKER_URL. It resolves retained paths to canonical repository IDs and reads repository metadata and deletion status. Beto’s current project.view gates every read; the dynamic can_delete flag comes from repositories.delete, reserved for Project administrators. Provider IDs remain private bindings.

A repository_deletion aggregate is keyed by canonical repository ID. It stores the requesting Party, Project and exact provider binding. DeleteRepository requires a command ID and expected deletion version; its snapshot, receipt and command reply commit together. Authority checks and GitLab calls stay outside that transaction. A recovery worker scans complete pending work, rechecks current Beto authority and the saved native repository’s Project, then uses the existing scoped GitLab management credential. Repeated calls address the same native ID. A transfer or revoked access records refused; transient failures leave pending. A new command at the observed version can restart a refused request. Acceptance records scheduled, including GitLab’s delayed deletion. Repository presence remains the last provider observation; acceptance does not assert physical removal. The process drains request, reply and recovery workers before closing its database.

HTTP contracts live in contracts/source_control/http_api/; the public surface is in the API service. Members keep Developer/Maintainer GitLab roles. No GitLab administrator credential enters this process. Repository creation, catalogue, content and fork commands still use the retained compatibility routes.

Public repository publication

application.publications.Publications owns publication consent for one canonical repository. The authoring repository remains inside its private Project group. An authorised SetRepositoryPublication command records the Party, canonical Project and repository, original provider binding, desired visibility and expected publication version. Its receipt and command reply commit with the aggregate. Current project.manage authority is required for publication and withdrawal; repository reads expose can_publish, desired visibility, state and version.

The states are pending, published, private and refused. Provider I/O runs after the consent transaction. The worker first reserves a private provider copy and records its opaque binding before copying content or enabling public access. A stable reserved topic identifies the intended copy if a create response is lost. A conflicting topic, provider identity, Project path or shared-group relationship prevents adoption. Canonical repository identity continues to name the authoring repository; the public copy is a binding of the publication aggregate.

Installation creates the separate public pharos-public root. Only the existing Source Control manager and the installation administrator may be direct members of that root. Unexpected membership, sharing or a private namespace at that path stops installation. Publication creates public subgroups using canonical Project IDs and repository kinds, with copies at pharos-public/<Project ID>/<kind>/<name>. Project display names, group shares and Project memberships are never copied into the public hierarchy. The authoring namespace and Run credential issuer keep their existing scope.

The Git adaptor copies heads, tags and Git LFS objects into a temporary bare repository. It uses verified HTTPS URLs, the mounted CA and the scoped manager credential. Credentials travel only in the child environment. Imported working trees, hooks, submodules, custom LFS transfers and credential helpers do not run. LFS endpoints are fixed to the two approved repositories. Complete LFS upload precedes the atomic ref push. A changed default branch is installed before obsolete refs are pruned, and public visibility is enabled last. Branch protections are removed only from the verified managed copy so it can follow source history rewrites; the private author's protections remain in force. The public copy has pipelines, runners and unrelated project features disabled.

Current authority is rechecked before sending new content and before enabling public visibility. Revoked authority, a missing source or a source moved to another Project withdraws the owned public copy and records refused. A new authorised command can retry. Explicit withdrawal sets only the owned copy private and remains durable during a provider outage. A removed public copy is reserved again before publication. Renames and moves retain the public provider identity. After a Project transfer, the new Project can still read canonical repository metadata. Its publication status stays private and pending until the old publication is withdrawn. It can then submit new consent, which rehomes the same private copy before publishing under the new Project's path. The saved previous copy Project and provider binding recover a lost transfer response.

Pending work is retried every five seconds. Published repositories are checked every thirty seconds; equal Git refs avoid a full copy. A published state records the last completed copy, so subsequent authoring edits can take one repair interval to appear. Provider failures retain durable work for retry. These server workers introduce no browser polling; the UI can request the current status on an explicit refresh.

The workload bounds temporary storage to 2 GiB, and each Git command to five minutes. Shutdown interrupts the owned Git process group and stops subsequent provider work before the database closes. Source Control requires git and git-lfs in its image, plus the existing GitLab HTTPS network access.

Page source

Source snapshot · Revision f928834c8.