# DD — GitLab groups and membership (4.1, 4.2) [Common design](dd.md) · [Tasks and completion status](adr.md) · [Source Control context](../../../contexts/source_control/README.md) ## Decision Source Control is the sole writer for managed GitLab Project groups and direct memberships. Beto Project owns canonical Projects, membership and access policy; it publishes desired facts and current authority. Beto does not call GitLab or duplicate Source Control's retry state. The temporary account writer remains the retained compatibility account workflow. It creates or links GitLab users and hands a verified provider association to Party Governance. Party Governance is the sole writer for that association. Source Control consumes it and never creates an account. | Operation | Active writer | Authority/result owner | | --- | --- | --- | | Create or link a GitLab account | Retained compatibility account workflow | Party Governance records the verified association; account workflow records provider progress. | | Create or rename a Project group | Source Control | Source Control records pending/applied/failed GitLab work and provider effects. | | Add, change or remove a direct group member | Source Control | Project supplies desired policy; Source Control records the GitLab effect. | Administrator credentials remain with installation/recovery tooling. Source Control receives a scoped runtime token for the closed group/member operation set. ## Canonical input and provider bindings The durable inputs are `ProjectChanged`, `ProjectAccessChanged`, `PartyRegistered`, `PartyAuthenticationChanged` and the two Party provider-binding events. Each has a distinct durable queue and receipt. `ProjectChanged` supplies complete material state; a current owner snapshot supplies separate freshness. `ProjectId` forms the stable child-group path. Membership uses `PartyId`. Waldur IDs, GitLab user/group IDs, usernames and paths remain opaque provider bindings. The Project event's external member ID is never treated as a GitLab account. The verified GitLab binding's external value is a numeric GitLab user ID. Missing or multiple bindings, missing eligibility and a user absent from GitLab are visible pending/conflict states. Account ownership is never inferred from email. `site_username` may be absent and belongs to the separate account-creation input, not group/member reconciliation. ## Group and role rules - Installation administration creates one private root group. It contains the Project groups and has no Project members. The runtime token cannot create it. - The child path is the canonical `ProjectId`. Its display name is the Project name followed by the ID, with GitLab-invalid characters replaced and length limited to 255 characters. - Rename changes the display name only. Source Control never deletes the group. - Under the temporary #1001 policy, both `developer` and `project-administrator` map to Maintainer (40). This permits ordinary members to create repositories and push to the protected default branch. The policy changes GitLab access only; Waldur roles and Beto membership administration remain unchanged. Unknown roles grant no access. Existing memberships are upgraded by periodic repair with fresh authority; stale authority retains the last applied level. See the [owner rules](../../../contexts/source_control/README.md#authority-freshness-and-revocation). - Direct members not desired by current policy are removed, except the group Owner. Closure retains the group and removes managed direct members. - Each Project group holds one subgroup for every repository kind Source Control resolves (`KINDS`), at `/`. They are made with the group, read once per pass, made again when one is lost, and never given members of their own: the group's members reach them by inheritance. Each kind group lets Maintainers create repositories; the Project group lets no one, so none is made beside the kind groups where the resolver would not find it, and a repository named like a kind cannot take the path of that kind's subgroup. All of them let only Owners create subgroups, so a repository is always at `//`, where the resolver looks. A group made before them, or changed by hand, gets them on the next pass. A refused subgroup or setting is reported and does not hold back access changes. A pass reads the group and all paginated direct members, plans a deterministic difference, applies additions/role changes, then removals. GitLab operations are idempotent. A later pass completes interrupted work and repairs manual drift. ## Freshness and revocation New grants and role increases require unexpired `ready` source evidence for the same Project revision. Pending, failed, stale or expired evidence retains only previously applied access, capped at its confirmed role. Current removals and role reductions remain effective. Party disablement, binding retirement and Project closure revoke managed membership immediately. This gives revocation a safe path while preventing a delayed account or binding from turning stale membership evidence into new access. Source Control persists the work state, attempts, pending/conflicting Parties, unresolved GitLab users and redacted failure class. Redacted counts are available from process diagnostics. ## Delivery and recovery Projection change, receipt and pending follow-up work commit before broker ACK. Provider calls run afterwards. RabbitMQ retries transient failures and transfers invalid or exhausted deliveries to the context's dead-letter queue only after confirmed publication. The periodic repair pass covers a crash after commit, GitLab outages and manual drift. Bootstrap starts consumers before a complete, paginated Project inventory read. Every item then uses `GetProjectIntegrationSnapshot`; changes during the load converge through the same revision rules. Duplicate events are harmless, older events are stale, and a conflicting payload at one revision is rejected. Repeating bootstrap rebuilds lost Project projections. Known Party bindings can be repaired through the singular owner query. Activation remains blocked because Beto has no complete canonical Project inventory or retained provider-binding replay. The singular Project/Party queries exist, but their production Metallo adaptors and an explicit Project retirement result are also pending. The installed deployment stays at zero replicas. ## Retained writers and handover Themelio's GitLab root actor (`compat/themelio/services/gitlab-root-actor`) exposed four operations that wrote the same objects under Themelio's own layout: | Route | What it did | After the handover | | --- | --- | --- | | `POST /waldur/project_create` | Creates the six `/` subgroups and writes the Waldur metadata. | Source Control creates `/` with one subgroup for each repository kind. | | `POST /waldur/add_project_member` | Finds the user by email and adds them to the six subgroups, with an invitation when none is found. | Source Control grants the role from the verified Party binding, by GitLab user id. No email lookup and no invitation. | | `POST /waldur/delete_project_member` | Finds the user by the stored email mapping and removes them from the six subgroups. | Source Control removes the member by GitLab user id. | | `POST /waldur/project_delete` | Deletes the six subgroups with their repositories. | Nobody deletes a group; closure removes the direct members and keeps it. | Creating or linking a GitLab user (`PUT /users/:username`, `PUT /users/:id/aup`) and the personal subgroups stay with the retained account workflow, as above; Source Control never creates a user. The four routes go together: both member routes look up the `/` groups that `project_create` makes and fail without them. There are no production users, and the new execution flow does not preserve Themelio's namespaces (decision on #759), so nothing is migrated. For each operation the order is: switch its legacy route off, enable Source Control for it, check it on the dev box, and only then remove the legacy code. Both are never on, and neither is off for longer than the check takes. | Operation | Check after the switch | | --- | --- | | Project group | A new Project has its `/` group and no `/` group appears. | | Add a member | The member is in the Source Control group, added by the worker, and no invitation is sent. | | Remove a member | The member is gone from the group and nothing else changed. | | Project removal | After closure the group exists with no direct members. | Status: retired (#1000). The four routes answer `410 Gone` and write nothing (`compat/themelio/services/gitlab-root-actor/src/compatibility/retired.ts`); only the Project detail read (`GET /projects/:projectSlug`) is left of the retained sync. Nothing in this repository called them. On dev boxes Source Control makes the groups and members of Beto's Projects (#941): on dev-vtheofil2 the seeded Projects have their `/` groups with their members, and no `/` group exists. With the routes retired, a new Waldur Project there got its group, its six kinds and its member as Maintainer; removing the member in Waldur removed them from the group and its kinds; the root actor received no `/waldur/*` request. Deleting a Project in Waldur does not close it: Beto lets only the operator close a Project ([Project context](../../../../beto/contexts/project/README.md)), so the group kept its members and closure through Source Control was not exercised. Where Source Control is still paused, no Project group is written, as before. ## Configuration and qualification The process owns `SOURCE_CONTROL_DATABASE_URL` and `SOURCE_CONTROL_BETO_BROKER_URL`. Its GitLab connection uses a scoped token file, trusted CA file, fixed HTTPS base URL, configured instance and pre-provisioned root group. Network policy admits PostgreSQL, RabbitMQ, DNS and HTTPS. Unit coverage exercises initial provisioning, account delay, membership changes, removal, Party disablement, binding retirement, event reordering, stale authority, failed work, restart repair and lost-projection rebuild. Real GitLab evidence for the worker, `reconcile` and the handlers is below. PostgreSQL and RabbitMQ evidence, and the production owner adaptors, must still be added for this integrated Source Control process before replicas can be enabled. ## Verification on a real GitLab Run on 25 October 2026 against GitLab 19.4.2, the version the repository pins, on a dev box installed from this branch. The installation is the one the context expects: the root group is made by an administrator, the worker is a plain user that owns it and cannot create top-level groups, and its token is the only credential it holds. The worker, `reconcile`, handler and creation runs gave the same results on 18.4.6. Method and commands: `TESTING.md`. | Script | What it shows | | --- | --- | | `qualify_source_control_groups.py` | The worker creates no top-level group (403) and cannot see or join a private group outside its root (404). A new Project gets a private group at `/`, named after the Project and its ID, with Developer and Maintainer from the two roles. A repeat changes nothing; a role change grants before it revokes; a member added by hand is removed; a group whose creation lost its answer is found, not duplicated. A rename changes the name only, so the id, the path and a repository's clone address are unchanged. Greek letters and an emoji are accepted, `/` is replaced and a 400-character name is cut to 255. An unknown GitLab user is reported and the others are granted. | | `qualify_source_control_handlers.py` | Discovery before login leaves members visibly pending; each binding then grants its member. Repeated snapshot and binding events write nothing. A grant GitLab made whose answer was lost is found by the next run, after a restart, without repeating it. A failure before GitLab acts is applied on redelivery. Stale evidence adds no access and fresh evidence does; a disabled Party and a retired binding remove access; closure removes every direct member and keeps the group; a late older snapshot restores nothing. | | `qualify_source_control_creation.py` | Run on 29 October 2026 against GitLab 19.4.2: 0 failed. The Project group the worker makes lets no one create a repository in it and its kind group lets Maintainers: a Developer is refused (HTTP 400) and a Maintainer creates a repository (HTTP 201) in the kind group. A root-group setting does not change the level of a kind group made afterwards, and the plain worker can change the level on its own group and back. | | `qualify_source_control_layout.py` | Run on 29 October 2026 against GitLab 19.4.2: 0 failed. A new Project gets its group and one subgroup for each of the six kinds in `KINDS` (private, named after the kind). Each kind group lets Maintainers create repositories and the Project group lets no one, and all of them let only Owners create subgroups; a second run changes nothing and sets nothing, so GitLab returns both levels in its group and subgroup records. The Project group's members hold their level in a subgroup by inheritance; a Developer is refused a repository (HTTP 400, namespace not valid) and a subgroup (HTTP 403); a Maintainer creates a repository at `//models/` (HTTP 201), is refused a repository directly in the Project group (HTTP 400, namespace not valid) and is refused a subgroup (HTTP 403). A subgroup deleted by hand is made again with a new id while the others stay the same; a Project group made by the previous version, which let Maintainers create repositories, is closed and gets its subgroups on the next run; levels changed by hand on the Project group and on a kind group are put back. | | `qualify_source_control_edges.py` | A member who has to lose access loses it although another wanted member's grant is refused (E7). A Project with 101 members is granted and read back in full (the member read has a page of 100). A group deleted by hand is made again at once with a new id, empty. A revoked token fails with a `GitLabError` 401, and a new token written to the same file works with no restart. | ### Observed limits These are what the real GitLab did, not decisions. Each needs an owner's answer. - A wanted member who already holds a higher access level by inheritance, such as an installation owner of the root group or the worker itself, makes `ensure_member` fail with HTTP 400 on every run. The run reports the failure and retries it, but every other step still runs: the other members are granted and access that has to go is removed (E4, E7). - Two concurrent runs that create the same group make GitLab answer 500 (a unique-index violation) instead of "already taken"; the worker treats it as an error and the next run settles it. - A blocked GitLab user is visible to the worker as `state=blocked` and is granted membership; only an absent user is unresolved. - A group deleted by hand is recreated empty with a new id and nothing records that it happened. - GitLab can answer a busy instance with 5xx, and repository creation can fail with `Failed to create repository`; both clear on a later run. - GitLab's default lets Developers create repositories in a new subgroup. Source Control sets each kind group's `project_creation_level` to Maintainer and the Project group's to `noone`; existing groups receive these settings on reconciliation. This is Source Control policy. Beto supplies canonical Project membership and roles. - A Maintainer could create a subgroup inside a kind subgroup that the worker had made before the owner-only setting (HTTP 201, 27 October 2026). A repository in a group one level deeper than `//` is not found by the resolver, so the Project group and the kind groups now allow only Owners to create subgroups (see above); a Maintainer is then refused (HTTP 403, 29 October 2026). Not covered here: PostgreSQL, RabbitMQ, the production owner adaptors and the initial load from Beto (#754), and the creation of repositories inside the `` subgroups (#758).