# Pharos Platform Acceptance Suite

Stakeholder-readable Gherkin scenarios run through Playwright Test and TypeScript
using `playwright-bdd`. This suite targets the working retained Themelio UI and
API-owned authentication restored by
[PR #976](https://github.com/pharos-ai-factory/pharos-platform/pull/976),
with canonical Project, repository and Run views from
`feat/canonical-project-run-ui`. The stack is `main` → #995 → #976 → that branch.
[ADR 0068](../docs/decisions/0068-platform-acceptance-tests.md) records the decision.

The base suite landed on `main` in
[PR #987](https://github.com/pharos-ai-factory/pharos-platform/pull/987).
The canonical UI branch adapts its Project identities, repository paths and
cleanup commands, and extends repository-category coverage.
Runs are **ad hoc**. Scenario generation, TypeScript checks and harness unit tests
run locally; browser scenarios qualify the configured deployment.

The separate [M18 demo suite](demo/README.md) covers the six user-guide video
journeys and produces a Markdown results table with masked screenshots and
explicit implementation gaps. The [k3s-m3 qualification report](reports/m18-demo-k3s-m3/report.md)
records the 4–5 November 2026 run against #1052.
The [CPU follow-up report](reports/m18-cpu-followup-2026-11-06/report.md)
records subsequent fixes, browser retries, CPU HPC on builder-m3 and KServe
qualification on k3s-m3.

## Deployment target

Run against a deployment containing `feat/canonical-project-run-ui` and its
#976 and #995 dependencies, with the matching API/backend. Set `baseURL` to that
UI's public origin, such as `https://themelio.localtest.me`.
Selected live scenarios require their UI, backend and fixtures to be available.

Record the revision actually installed at that origin in `deployedRevision`.
Updating the test branch does not deploy the application. Earlier results remain
evidence for their recorded deployment; qualify the restored UI in a fresh run.

## Prepare and run

Use the repository's pinned pnpm and Node toolchains:

```sh
pnpm install --frozen-lockfile
pnpm exec playwright install chromium
pnpm acceptance:check
pnpm acceptance:list
```

Copy `acceptance/config.example.json` to an ignored location, such as
`.local/acceptance/target.json`. Set the installed public origin and its actual
deployed revision. Paths inside this JSON resolve relative to the JSON file.
Remove optional settings for scenarios you will not run. Public smoke checks
need only `baseURL` and `deployedRevision`.

```sh
ACCEPTANCE_CONFIG=.local/acceptance/target.json pnpm acceptance:smoke
ACCEPTANCE_CONFIG=.local/acceptance/target.json pnpm acceptance:run --grep @authentication
ACCEPTANCE_CONFIG=.local/acceptance/target.json pnpm acceptance:run --grep @repositories
ACCEPTANCE_CONFIG=.local/acceptance/target.json pnpm acceptance:run --grep @actions
ACCEPTANCE_CONFIG=.local/acceptance/target.json pnpm acceptance:run --grep @hpc
pnpm acceptance:report
```

From a workstation, a dev box's `*.localtest.me` origins answer only through an
SSH SOCKS tunnel. Open one and point the browser at it; the variable accepts only
a loopback `socks5://` or `http://` proxy without credentials:

```sh
ssh -N -D 127.0.0.1:1080 <dev box>
PHAROS_BROWSER_PROXY=socks5://127.0.0.1:1080 ACCEPTANCE_CONFIG=.local/acceptance/target.json pnpm acceptance:smoke
```

The onboarding drivers `scripts/authentication_live_browser.mjs` and
`scripts/platform_authentication_live_browser.mjs` read the same variable.

All acceptance browser login paths allow ten seconds for onboarding and provider
updates to settle before the next action. This includes the shared researcher,
first-use policy, external federation, Support and hosted Chat paths. Their
session, navigation and visible-readiness assertions still run after that pause.

`acceptance:run` with no filter runs every **implemented** scenario. A missing
credential, fixture, permission or unavailable service fails the selected scenario.
`acceptance:list` needs no deployment or credentials. Planned specifications are
listed separately below and do not contribute passes or skips to the report.

The Playwright HTML report is `playwright-report/acceptance/index.html`; the
stakeholder Gherkin report is `playwright-report/acceptance/scenarios.html`.
Machine-readable results, individual durations and errors are in
`test-results/acceptance/results.json`. Each scenario records its origin and
declared deployed revision. Record the test checkout revision and any local diff
alongside these reports. A report qualifies only the named deployment and selected
scenarios. Reports are private local outputs. Each invocation replaces the previous
report; copy both report and result directories to a dated evidence folder before
another run.

## Accounts, data and trust

- Use a dedicated, disposable researcher account, with AUP already accepted and
  GitLab onboarding complete. `credentialsFile` is a regular file with mode
  `0600` containing `{"username":"...","password":"..."}`. `issuer` is the
  exact enrolled Keycloak issuer. The login steps verify that issuer before
  entering credentials. Academic/Google federation with interactive MFA remains
  a separate manual check; a local AAI pass establishes only that configured path.
  After sign-in, the UI must reach the dashboard automatically as onboarding
  becomes ready. The suite observes readiness without refreshing the page.
- `namespace` is the canonical UUID of an existing disposable Project returned by
  `GET /api/v1/me/projects`, in which the test account can create repositories. Replace the example UUIDs with the target's canonical IDs.
  `gitlabRootGroup` defaults to `pharos-projects` and must match the deployed
  `SOURCE_CONTROL_GITLAB_ROOT_GROUP`. Source Control must have provisioned the
  Project's category groups and membership before repository checks run.
  The suite uses unique
  `acceptance-*` names, private repositories and minimum-scope Git write tokens.
  It revokes its tokens, requests deletion of its repositories and ends its own
  sessions during teardown. The account must have Project administrator rights.
  Cleanup resolves the canonical repository ID, submits the Source Control
  deletion command and waits for its recorded `scheduled` outcome. Uncertain
  replies retain the same command key, repository ID and version. GitLab may
  complete physical deletion asynchronously under its retention policy.
  A cleanup failure fails the test and identifies the owned resource to recover.
  Keep this account and namespace exclusive to one run, including runs from other
  checkouts. The suite uses one worker and no automatic retries.
- `publicRepository` identifies a stable, public fixture visible in the catalogue;
  its `namespace` is also a canonical Project UUID.
  Public smoke allows a genuinely empty catalogue; the fixture scenario proves
  that an actual repository can be discovered and opened.
- HTTPS validation stays enabled. For private installations, trust the verified
  CA on the workstation or set `NODE_EXTRA_CA_CERTS` to its PEM before starting
  Node. `caFile` supplies the same trust to Git. `browserSpkiFile` optionally
  supplies independently verified certificate-key pins to Chromium. Obtain trust
  material through the deployment's authenticated administration path.
- SSH scenarios require an explicit `ssh` object with `host`, `user`, `port`,
  `identityFile` and `knownHostsFile`. They use OpenSSH with strict host checking,
  an explicit key, no agent forwarding and no inherited SSH configuration.
  They establish member login; key provisioning and Slurm job submission need
  their own checks.
- Traces, screenshots, videos and automatic full-page failure DOM snapshots are disabled
  because sign-in and token issuance expose credentials. Passwords, tokens and
  browser storage state are never attached to reports. The failure-snapshot
  setting is verified against pinned Playwright 1.63 and must be reviewed on upgrades.

For personal dev boxes, the existing [identity fixture tool](../scripts/authentication_live_provider.py)
and [live onboarding driver](../scripts/authentication_live_browser.mjs) can prepare
a disposable account. Their administrative setup and final account removal are
separate from the acceptance run. Use their documented ownership checks.

Repository checks use the public application APIs for cleanup. They never write
owner tables or use administrator credentials in application containers. Browser
steps make real requests; they do not replace routes with mocked responses.
Unit checks of the harness itself provide separate, local evidence.

The `@history` repository scenario pushes two real README commits, checks their
order and full identifiers, opens the earlier file content, reloads its revision
URL and returns to the current revision. It uses the same owned repository and
token cleanup as the other Git scenarios.

The shared file browser also has a separate local fixture lane:

```sh
pnpm --dir compat/themelio install --frozen-lockfile
pnpm exec playwright test --config acceptance/browser/playwright.config.ts
```

This lane bundles the actual file-browser components and serves deterministic
API responses on loopback. It checks pagination, exact-revision URLs, browser
back/forward navigation and visible errors without deployment credentials.
These fixture results do not establish live provider or publication behaviour.

## Coverage of the requested journey

“Implemented” means an executable scenario exists. Live evidence is established
only by its report against a prepared target.

| ID | User capability | Current coverage |
| --- | --- | --- |
| P01 | Homepage, catalogue and public resources | Implemented: smoke and known public repository |
| P02 | AAI / academic / Google login | Implemented: configured Keycloak AAI; external federation remains manual |
| P03 | Proposal or playground request and waiting | Planned: retained proposal pages still use demo local storage |
| P04 | Approval/rejection and notification | Planned: real reviewer workflow and notification contract required |
| P05 | Dashboard and resource creation | Implemented: dashboard and all six repository categories |
| P06 | Issue Git API tokens | Implemented: UI issuance, real Git use and revocation |
| P07 | Upload repository files | Implemented through Git push; a separate browser-upload path is not covered |
| P08 | Browse repository files | Implemented: pushed sample, paginated commit history and exact-revision contents |
| P09 | Fork a public repository | Planned: isolated source/destination fixture and cleanup |
| P10 | Commit actions.yaml and see actions | Implemented: commit a small batch definition and inspect the menu |
| P11 | Previous action runs | Planned: seeded run and real execution environment |
| P12 | Run configuration and logs | Planned: same prerequisite |
| P13 | Launch a new action and inspect it | Planned: workers and Site allocation must be enabled |
| P14 | Completion notification and detail link | Planned: exact run correlation and separate UI/email checks |
| P15 | SSH to Slurm login node | Implemented: authenticated member and remote username |

The [allocation specifications](planned/resource-allocation.feature),
[fork specification](planned/repository-fork.feature) and
[execution specifications](planned/execution.feature) retain the agreed intent
for stakeholder review. Their steps have no executable implementation yet.
Current provider/infrastructure scripts provide supporting evidence for their
respective boundaries. Execution scenarios require a deployment with its workers
enabled and a provisioned Site allocation.

## Writing and extending scenarios

Keep one business outcome per scenario and name actors and notification channels
explicitly. `Given` establishes prerequisites, `When` expresses an action and
`Then` asserts an observable result. Keep selectors, API shapes and provider
details in `steps/` and `support/`. A scenario must work independently; setup may
use supported APIs, while the user action under test uses its actual interface.

Use `@P01`–`@P15` for coverage, capability tags for selection and `@writes` or
`@external` for operational scope. Add a runnable scenario under `features/` only
with implemented steps and explicit prerequisites; generation rejects missing
steps. Keep future specifications under `planned/`, with the gap recorded above.
Use British English and the same business terms across product and tests.

As the replacement platform exposes each capability, retain valid Gherkin
outcomes and replace the appropriate UI/API helpers. Review intentional behaviour
changes with stakeholders. Measure repeated run durations, setup time and failure
causes before proposing a CI subset or release gate.