# DD — Project repositories and action-file reads (#759) [Common design](dd.md) · [Tasks and completion status](adr.md) · [Source Control context](../../../contexts/source_control/README.md) · [GitLab groups and membership](gitlab-groups.md) Here a Project is a Pharos Project, whose ID Beto gives. What GitLab calls a project is called a GitLab project or a repository. ## Decision Source Control answers three questions for execution: which repository a URL or a `{type}/{namespace}/{name}` path names, whether it belongs to the requested Project, and what `actions.yaml` says at a branch, tag or commit. It answers with canonical repository IDs and the member's current access, and never shows one Project's repository to another. The domain and application name repositories, Projects, kinds, refs, commits and files only. A repository's provider identity is an opaque binding `(provider, instance, external_id)`, like Beto's provider bindings. Everything GitLab-specific lives in one adaptor behind the `RepositoryProvider` port: API paths, GitLab project numbers, the group layout, redirects, base64 file encoding, clone-URL rules and errors. The group/member slice of #739 keeps its GitLab-shaped domain (`domain/gitlab.py`, the `Provisioner` port); this change does not alter it. Decisions recorded with #759 (24 October 2026): - The owner side was delivered first; the selector, request-host and execution-worker wiring followed in #796. - Namespaces are canonical Pharos Project IDs. - The layout `///` is agreed and owned by Source Control. Repository creation goes through Source Control and uses the same convention; the GitLab path stays a provider detail, and the canonical repository ID stays stable across a rename or transfer. - There are no production users, so existing Themelio repositories need no automatic migration or compatibility work. They stay in Themelio; examples worth keeping are recreated or moved into a Pharos Project by hand. Preserving the existing namespace behaviour is not required for the new execution flow. - Repository changes are not published as integration events until a context needs to consume them (below). ## Identity A repository gets a random UUID the first time Source Control sees it, stored with its binding in `pharos.source_repositories` (Source Control migration 1). The binding is unique, so a rename or transfer keeps the ID. A deleted repository is marked `missing` and its row is never removed; the runtime role has no `DELETE`. A new repository created later at the same path is a different repository with a new ID. The table is a context-owned record, not an aggregate. Reads record what they observe: each change of address, location, default branch or state increments the row's revision, and an answer that was asked for earlier than the saved one never replaces it. Concurrent first sightings give one row. ## Addresses and namespaces A Project repository is addressed as `(project_id, kind, name)`. Kinds are Beto's `RepositoryKind`: `models`, `datasets`, `spaces`, `recipes`, `upskillings` and `repos`, one subgroup each in the Project group (#833); migration `0002_repository_kinds.sql` widens the stored check from the first four. Execution Core still runs actions only from models, spaces and plain repositories. Names follow Themelio's rule. The namespace of the retained selector is the canonical Project ID, which is also the path of the Project's GitLab group and never changes. A namespace that is not a Pharos Project, such as Themelio's `{type}s//` layout, answers "not found", exactly as an absent repository does. Source Control never touches that layout, and the new execution flow does not run from it. Project slugs follow when Beto defines them. ## Access A member sees a Project's repositories when the Project is `active`, the member holds a contract role (`developer` or `project-administrator`) in the currently authorised membership, and their Party is `enabled` and authentication-eligible. The membership is the one that grants provider access (`capped_members`): with stale or pending Project evidence only access previously applied to the member's current account remains, so a new member, or one whose account changed, is withheld. A member without access causes no provider call. ## Queries `Repositories` (application) and `RepositoryReads` (the adaptor façade other contexts call): | Query | Answer | `None` means | | --- | --- | --- | | `repository(member, kind, namespace, name)` | `Repository(id, Project, kind, name, state)` | absent, hidden, in another Project, or not a Project namespace | | `project(member, namespace)` | the Project ID | not a Project the member sees | | `resolve_url(project, url)` | the canonical ID | not one of this Project's repositories | | `read_actions(member, project, repository, ref, expected_digest)` | `ActionsFile` | repository gone, hidden or moved away, or no `actions.yaml` | | `commit_of(member, project, repository, ref=None)` | the commit the ref names now; without a ref, the default branch's | repository gone, hidden or moved away | | `describe(repository)` | the last known `Repository`, including `missing` | unknown ID | An old or differently spelt path leads to the repository when it is still in the same Project. Each read asks the provider where the repository is now, so a transfer moves it to the new Project and the old one loses it. `describe` asks no provider and checks no member: a Run keeps its repository after a rename, transfer or deletion, and the caller applies the Run access policy. Rejections: `ref_not_found` for an unknown branch, tag or commit, or a repository without a default branch when `commit_of` is asked for it; `file_changed` when the file differs from `expected_digest`; `file_too_large` above 256 KiB. Every outage — the database, the provider or state that cannot be restored — is `SourceControlUnavailable`; nothing was decided and the caller retries. ## Run preparation response `read_actions` resolves the ref to its commit and reads the file at that commit. It returns the repository, the Project, the ref asked for, the commit, the file's SHA-256, its size and its bytes. A launch cannot use a different file than the one selected: it reads again at the selected commit, which is immutable, or passes the selected digest and is refused when the branch moved to a different file. `commit_of` answers the commit a branch names now, so a Run pins its other repositories without reading a file (#767). It applies the member's access like the other queries. A Pipeline's later steps pin when they start; when the member who started the Run has left the Pharos Project, the query answers `None` for them. Accepted Runs continue when their initiator leaves (#755), so those steps need the Pharos Project's execution authority rather than the initiator's own access; that decision belongs to #766 and #767. ## GitLab adaptor - Layout: `///` under #739's Project group, as agreed. `external_id` is the GitLab project's number. Source Control's reconcile makes the kind subgroups with each Project group (#758); creating repositories in them goes through Source Control with this layout and is not part of this change. - Clone URLs: HTTPS only, on the configured host and port, under the configured base path, without user information, query or fragment, with at most one trailing `/` and one `.git`. - GitLab 19 answers a renamed or transferred repository's old path with `301` to `/api/v4/projects/` on its external URL. The transport never follows a redirect, because the token would go wherever the redirect points; the adaptor takes only the GitLab project named in it and asks the configured GitLab once more. This applies to the group/member worker's transport as well; #792 brings the same change and its regression test into #739. - A GitLab project scheduled for deletion, or one left outside the layout (GitLab renames a group scheduled for deletion to `-deletion_scheduled-`), is gone. - A file's size is asked with `HEAD` before it is downloaded. The content must be base64, match its size and `content_sha256`, and come from the commit asked for. - Reads use a separate read-only token, `SOURCE_CONTROL_GITLAB_READ_TOKEN_FILE` (Reporter, `read_api`), never the group-management token. The dev-box run used a group access token on the root group. Observed on GitLab 19.4.2: a transfer is answered before it shows, and for a few seconds the old and new paths both resolve; a new member of the root group (the read token's bot) sees existing repositories only after GitLab's background authorisation refresh. Reads in those windows answer from what GitLab shows, and the next read converges. ## Consumers and composition `metallo.adaptors.source_control_actions` implements Workload Portfolio's `RepositoryActions` port over `RepositoryReads`: it returns the parsed document and its exact GitLab commit in `ActionsRead`, preserving the input-pinning contract from #767. It parses UTF-8 `actions.yaml` with `yaml.safe_load` (PyYAML becomes a runtime dependency), answers an empty file as an empty document, rejects invalid YAML as `invalid_actions_file`, and leaves a document that is not a mapping to Workload Portfolio's own validation. Before construction, an event pass rejects cycles and limits nesting to 32 collections and expanded size to 10,000 values, counting mapping keys, alias uses and merge inputs. This bounds the parser's recursive composition and merge expansion; ordinary shared anchors and merge precedence are preserved. The constructed document is also checked before Workload Portfolio walks it. Outages become `RepositoryUnavailable`. The process that hosts these reads mounts Source Control's database credential and the read-only token. Execution Core and Workload Portfolio see only their ports. ## Follow-ups - Done in #796: the retained selector ports, `RepositoryActions` and `SourceVersions` are composed over `repository_reads` in `metallo-requests` and the execution worker (see below). The overlay that connects a GitLab still has to set the connection values and provision the read token before those processes start. - Publish repository changes when a context needs them. Before another context relies on a local repository projection, publication must be reliable, through a Source Control outbox, and the projection must have a way to load or rebuild. Each change already increments the row's revision. - Repository creation with the agreed layout (#758); Project slugs as namespaces. - Run-scoped credentials (#762), which must outlive the initiator for Runs that continue after they leave. ## Composition in the processes (#796) `metallo.adaptors.source_control_owners` builds, over one `RepositoryReads`: `RunRepositories` (Execution Core's `RepositoryDirectory`), `ActionRepositories` (Workload Portfolio's), `SourceControlVersions` (`SourceVersions` over `commit_of`) and `SourceControlActions`, which now returns the commit the file was read at (#767). `metallo-requests` and the execution worker open Source Control's database beside their own and read GitLab with the read-only token; without the `SOURCE_CONTROL_GITLAB_*` connection values they refuse to start, as Source Control's own process does, and the refusing owner stubs for these ports are gone. Each port's "retry later" error carries a Source Control outage; Source Control alone decides what a member may see. Both Deployments mount the three credentials and get HTTPS and DNS egress for GitLab; the URL, root group and instance come from the connecting overlay. ## Verification Completed on 24 October 2026: - **146 new unit cases** with a provider in memory and the handler harness's access facts: canonical IDs by path and URL, rename and alias, transfer, deletion and a reused path, cross-Project URLs and aliases, every loss of access without a provider call, stale evidence, personal and malformed namespaces, branch, tag and commit reads, a branch moving between selection and launch, unknown refs, absent and large files, outage translation, YAML rules, the GitLab adaptor's layout, URL, redirect and file checks, read-token settings, and a transport that returns a redirect instead of following it. - **8 PostgreSQL cases** with the restricted runtime role: minting, unchanged and changed answers, older answers, missing and restored repositories, concurrent first sightings, no `DELETE`, corrupt rows, and the queries end to end over the real database with access facts written by the Source Control handlers. - `pnpm verify` (uv Python 3.14) passed with **2,719 Metallo tests**, strict Mypy, architecture checks and installed-wheel imports of the new modules. - `pnpm test:integration`: the 8 new cases pass in the Metallo lane (347 passed). The lane fails five existing cases in `source_control.py` and `pipeline_delivery.py` that fail the same way on the #739 base (`d2b4e1886`). - `scripts/qualify_source_control_repositories.py --target dev-dkats` passed **20/20** against GitLab 19.4.2 on the dev box (10.0.1.23) with a group access token (Reporter, `read_api`): the same ID by path and clone URL, another Project's URL refused, a non-member refused, branch/tag/commit reads, digest, a moved branch, the selected commit, unknown ref, missing and large files, rename with old and differently spelt paths, transfer between Projects, refused writes with the read token, and deletion with the record kept as missing. Its groups and tokens were removed afterwards. The wired-process test was completed on 26 October 2026 on `metallo-vasilis-2` against GitLab 19.4.2, PostgreSQL 18 and RabbitMQ 4.1. The [process smoke runner](../../../../../dx/metallo/README.md#source-control-process-smoke) starts the production request host and execution worker with Source Control's restricted database role and Reporter `read_api` token. A dedicated loopback HTTPS bridge verifies the fixture CA and forwards reads to the installed GitLab. - Repository paths and clone URLs resolve to the same canonical ID. An outsider and a URL from another Project are refused; the read token cannot create a branch. - The request host lists `actions.yaml` through the actual broker, accepts a launch and replays its saved decision with the same input commit. - The execution worker plans an initially unpinned saved Workload and persists the exact GitLab commit. After advancing the branch, the original Run keeps its old commit and the next Run uses the new commit, with the repository ID unchanged. - Both processes drain successfully. All 21 targeted Source Control and request PostgreSQL/broker regression cases pass on the combined branch. Permission/provisioning decisions and initial Beto facts use fixture owners; Source Control reads, GitLab HTTPS, broker delivery and persistence are real. Native dispatch, real Beto enrolment, browser/UAT and a GitOps rollout are outside this smoke test. Production activation still follows the owner and #771 gates.