# DD — Repository action pipelines (8E)

[Common design](dd.md) · [Tasks and completion status](adr.md).

"Run all in order" runs every action of a repository branch's `actions.yaml`, top
to bottom, as one Pipeline. 8E connects the two existing halves:
[2D](workload-creation.md#repository-actions) turns actions into saved Workload
revisions, and [8A–8C](pipeline-records.md) run an ordered list of Workload
revisions.

8E has two parts: the [internal start](#flow), and the [public start, read and stop
requests](#public-requests) a member or the browser uses.

## Flow

1. `StartRepositoryPipelineCommand` names the member, Project, repository, branch,
   new Pipeline ID and optional parameter values per action.
2. `RepositoryPipelines.start` checks current Project access.
3. The `RepositoryActionWorkloads` port prepares every action. Workload Portfolio
   reads `actions.yaml` once, keeps its declaration order and creates, reuses or
   revises each action's Workload. Each action gets a stable command identity
   derived from the request.
4. Each prepared action becomes a step: its exact Workload revision, environment
   (`slurm-job` → `slurm`, Kubernetes actions → `k8s`), family and service kind.
   Batch steps advance on success; services advance once healthy with an endpoint.
5. The existing internal Pipeline start checks the revisions through the real
   Workload owner adaptor and saves the Pipeline, receipt and coordinator work.

```mermaid
flowchart TD
    C["Run all in order<br/>repository, branch, values"] --> R["RepositoryPipelines.start"]
    R --> A["Check current Project access"]
    A --> P["Workload Portfolio prepare_all<br/>read actions.yaml once"]
    P --> W[("Workload Portfolio PostgreSQL<br/>one Workload per action")]
    P --> S["Ordered steps with environments"]
    S --> O["Pipeline start<br/>checks exact revisions"]
    O --> D[("Pipeline Coordination PostgreSQL<br/>Pipeline, receipt and work")]
    D --> K["8B coordinator launches each step"]
```

Workload preparation commits in Workload Portfolio before the Pipeline transaction,
so owner I/O stays outside Pipeline transactions. Every step is idempotent: a
retried command reuses the same Workload revisions and replays the Pipeline
receipt. A retry hands Workload Portfolio the revisions its first acceptance saved,
and each is kept while it still defines the same work, so another start that revised
the shared Workloads in between cannot turn the retry into other work. If
`actions.yaml` changes between a request and its retry, the retry returns
`idempotency_conflict` instead of starting different work.

## Public requests

The API exposes three operations, the path a single action's launch already takes:
an owner-enrolled route in `contracts/pipeline_coordination/http_api`, a typed request
to `metallo-requests`, and the use case in `application/member_pipelines.py`.

| Operation | Route | Identity |
| --- | --- | --- |
| Start | `POST /api/v1/{type}/{namespace}/{repo}/pipeline/{branch}` | `Idempotency-Key` names the new Pipeline; `If-Match: "0"` |
| Read | `GET /api/v1/pipelines/{id}` | None; current `pipelines.view` access |
| Stop | `DELETE /api/v1/pipelines/{id}` | `Idempotency-Key` names the stop; `If-Match` is the version last read |

- A start finds the repository through the directory a launch uses. A hidden or absent
  repository is `not_found`, and nothing is prepared.
- The branch is one percent-encoded path segment, so `feature/train` is sent as
  `feature%2Ftrain`; the generated web clients encode every path value.
- The start and stop receipts leave out the arrival time, as a Run's stop does, because
  a retry repeats the command identity and not the time. A retry replays the saved
  decision. A start retried after `actions.yaml` changed is `idempotency_conflict`.
- The reply commits with the decision in Pipeline Coordination's database and the
  request host relays it. An owner outage saves no receipt and answers 503 for retry.
- A read returns each step in execution order with its action name, status, and the
  Run it started. The Run state it carries is the one the Pipeline last observed, which
  stops changing once the Pipeline finishes; the jobs and deployments routes give the
  current one.

`metallo-requests` answers all of this owner's requests, including the internal query
for accepted Pipelines, with its own database and request-user credentials. The
coordinator keeps executing steps and keeps this context's Party and Project view,
which these requests read. Browser updates follow with 8F; until then a client reads
the Pipeline on demand.

## Ownership

- **Pipeline Coordination** owns the use case and its
  `RepositoryActionWorkloads` port. Steps keep only Workload references.
- **Workload Portfolio** owns reading actions, their conversion and the Workloads.
- [`adaptors/pipeline_actions.py`](../../../adaptors/pipeline_actions.py) connects
  the port to Workload Portfolio. It maps owner outages and corrupt state to
  `ActionsUnavailable` for retry; business rejections such as an unknown action or
  a missing required value pass through unchanged.

Every run of the same branch shares its action Workloads; each run is a new
Pipeline with its own step identities.

## Limits

- `actions.yaml` has no Kubernetes batch type. A branch can mix Slurm batch steps
  and Kubernetes services; Slurm execution is part of 6B.
- Single-action execution is unchanged.
- Reading `actions.yaml` uses the test-only repository owner until the GitLab
  repository connection is available.

## Verification

Completed on 21 October 2026:

- **13 use-case cases** cover ordered steps, environments, advancement
  conditions, stable step identities, retries, conflicting retries, access,
  rejections and invalid owner evidence.
- **4 cross-context cases** run Workload Portfolio's real preparation and
  Pipeline Coordination's real start together, through the production adaptors and
  in-memory storage: owner checks, retries, repeated runs and outages.
- **5 `prepare_all` cases** cover declaration order, one read, stable per-action
  commands, per-action values, unknown actions and empty files.

- `pnpm verify` passed with **1,827 Metallo tests** and strict Mypy over **297
  source files**. `pnpm test:integration` passed with **284 Metallo PostgreSQL
  cases**. The two new cases use both context databases with restricted roles: a
  branch starts one Pipeline over saved Workloads and replays its receipt, and four
  concurrent runs of one branch share the same Workload revisions.
- `make metallo-identity-test TARGET=dev-NAME SSH=your-dev-box` passed **1,589
  cases** on the dev box, including all 8E cases, and removed its namespace.

### Native Kubernetes verification

On 21 October 2026, `make metallo-pipeline-test TARGET=dev-NAME SSH=your-dev-box`
passed every [8D scenario](pipeline-verification.md#scenarios) and the new
repository-action scenario:

- A test-only repository owner read `actions.yaml` at the seeded branch's latest
  commit from the installed GitLab over HTTPS Git with the scoped token.
- Its `app` (`k8s-deployment`) and `serve` (`k8s-inferenceservice`) actions became
  Workload Portfolio Workloads, and one Pipeline started over them. Repeating the
  start command replayed the same outcome.
- The hosted coordinator and execution worker read those Workloads from Workload
  Portfolio through its restricted role. Each step had exactly one launch and Run.
- A real application Deployment and KServe InferenceService became healthy with
  endpoints, and the Pipeline succeeded. The application and the model answered
  private requests; both Attempt volumes held the branch's latest files.
- Cleanup removed the services, KServe resources, temporary GitLab access and
  repositories, and the owned namespaces.

Not run: the real GitLab repository owner, real permissions and Slurm actions.