# MCP laboratory

The optional `mcp` profile activates the [experimental API-owned MCP gateway and
OAuth broker](../../../services/api/services/mcp/README.md) on a development box.
It requires local AAI and `APP_ENV=development`. Public and distributed
deployments do not select this profile.

For the current ARM64 integration laboratory, deliver the committed snapshot:

```sh
make -f dx/arm64-singledrive/Makefile deploy \
  TARGET=dev-vasilis-m3 SSH=k3s-m3 DEPLOY_RUNNER=builder-m3 \
  PROFILES=mcp,kserve,zammad
```

`deploy` records a changed profile selection and prepares the edge, DNS and Argo
bootstrap rights. After activation, ordinary `reconcile` delivers new committed
source. `reconcile` alone does not change profiles.
The image catalogue builds `mcp-gateway`, `mcp-broker`, `aai-administration` and
`database-admin` from the delivered SHA and supplies their immutable digests.
Keep the normal delivery health gates.

| Purpose | Address |
| --- | --- |
| MCP resource | `https://mcp.localtest.me/mcp` |
| Broker issuer | `https://mcp.localtest.me/` |
| Resource metadata | `https://mcp.localtest.me/.well-known/oauth-protected-resource/mcp` |
| Broker discovery | `https://mcp.localtest.me/.well-known/openid-configuration` |
| Broker keys | `https://mcp.localtest.me/jwks` |
| AAI callback | `https://mcp.localtest.me/interaction/callback` |
| Private API origin | `https://api-auth.pharos.svc.cluster.local/` |

The host edge retains the public Host and replaces forwarding headers.
The gateway and broker accept only `mcp.localtest.me`; their longer development
hostname is refused. Cluster DNS resolves the enrolled name to the node edge.
The gateway fetches the canonical broker JWKS through the node's HTTPS listener
on port `8443`. Its fixed connection-port setting preserves the public URL, Host
and TLS verification name; it changes neither OAuth issuer nor audience.
The profile admits outbound HTTPS only to the delivered node addresses and the
private API listener. The private `/internal/mcp/identity` route is absent from
the public API catalogue.

The broker's dedicated `pharos-mcp` AAI client is appended only to the local
laboratory registry when this profile is selected. A person must first complete
ordinary Pharos sign-in and AUP. MCP delegates to their existing Party and
permissions. No operator Parties are enrolled by this profile.

The [AAI registration source](../../../devops/aai/mcp-lab-client.json) declares
these two exact AAI-to-broker redirect URIs:

- `https://mcp.localtest.me/interaction/callback`
- `https://mcp.staging.pharosfactory.ai/interaction/callback`

The local AAI reconciles this allowlist for the selected laboratory profile.
For external Pharos AAI, request a dedicated confidential OIDC client with the
same allowlist, authorisation-code flow, PKCE S256 and the declared
`profile`, `email` and `voperson_id` scopes. The administrators must provide the
client binding privately before external-AAI activation. This registration
request is pending; the laboratory continues to use local AAI. The staging
callback is reserved for a future deployment and does not enable staging ingress
or a staging broker.

See [client setup](clients.md) for Codex, Claude Code and OpenCode. The additional
`pharos-mcp-lab-smoke` client has the exact callback
`http://127.0.0.1:18743/callback`. All four clients are public native clients,
require PKCE and use explicit consent. Dynamic registration is disabled.
Their loopback redirects receive the broker's response in the local coding
client; the AAI redirects above return the upstream sign-in to the broker.
`base/config.yaml` holds their public enrolments; its values are copied into
pod-template annotations so a change rolls both processes.

## Administration and recovery

The bootstrap-owned `mcp-material` Secret preserves independent Ed25519 gateway
and RS256 broker keys, two cookie keys and separate migration/runtime passwords.
The administration hook persists these before creating derived Secrets. Retrying
an interrupted sync reuses them. Back up this Secret and the `mcp_oauth` database
together; deleting the material breaks existing grants and sessions.

The hook uses infrastructure administration only to prepare the dedicated
database and roles. It applies the exact revision's migration as `mcp_migration`
and gives `mcp_runtime` only CONNECT, schema USAGE and the required table rights.
It refuses foreign role memberships, database ownership and runtime access to
another database. The running broker receives no migration/admin credential.
The gateway receives no database credential. Runtime ServiceAccount tokens are
disabled.

The hook publishes only public verification keys into `pharos/api-mcp-public`.
Gateway and broker trust includes the local edge CA and the private API CA;
certificate and hostname verification remain enabled.
The API and MCP Applications share root wave 3. The API-auth credentials hook
publishes its trust before API readiness; the MCP hook then publishes the public
keys the API needs. Neither hook waits for an application HTTP response.

Key rotation requires a separate reviewed enrolment and restart procedure:
publish new verification keys before changing signers and retain retiring keys
for outstanding tokens. The initial laboratory does not rotate material on sync.
Removing the profile stops selecting its Application; follow the normal optional
service retirement procedure and preserve its database and material before removal.

## Verification

Run the focused deployment boundary tests and the real provider client fixture:

```sh
uv run --project devops --frozen python -m unittest discover \
  -s dx/common/tests -p test_mcp_lab.py
pnpm --filter @pharos/mcp-broker build
node devops/mcp_lab/verify_clients.mjs
uv run --project devops --frozen python devops/mcp_lab/verify_database.py
```

The provider fixture checks all four committed client registrations, exact
callbacks, issuer response binding and disabled DCR. It uses disposable in-memory
provider state. It does not establish live AAI or business-operation success.
The database fixture requires Docker and `psql`. It starts an isolated copy of
the pinned PostgreSQL image and checks migration-role ownership, runtime
permissions, state preservation on retry and rejection of foreign authority.
Full `pnpm verify`, the owning MCP/Party infrastructure lanes and a deployed
PKCE/consent/tool-read smoke remain required at the delivered SHA.