# ADR-0002: Architecture Documentation Update

## Status

Proposed

## Context

The architecture documentation under `docs/product/architecture/` has fallen out of sync with the codebase. Key problems:

- **Wrong service names**: References `themelio-web`, `themelio-web/api`, `themelio-gitlab-client` — actual paths are `services/web`, `services/web/src/app/api/`, `shared/gitlab-client`
- **Deprecated services undocumented**: `gitlab-secure-proxy` no longer exists on disk — it was replaced by `services/gitlab-root-actor` (Fastify, port 3001 — user lifecycle, impersonation tokens). `k8s-root-actor` (Fastify, port 3002 — inference/space deployments) is also undocumented
- **Authentication page describes a deprecated flow**: GitLab Omniauth + OpenResty Lua proxy is the old approach. Actual auth: NextAuth v5, OIDC provider, `gitlab-root-actor` provisioning, Valkey session, middleware token injection
- **Flows page is a monolith**: `flows/overview.md` contains 7+ mermaid charts mixing implemented and aspirational features with no distinction
- **Obsolete page**: `ai-assisted-development.md` references Django and VSCode roocode plugin — both gone
- **CLAUDE.md is stale**: Still references `gitlab-secure-proxy`, "Django/Next.js transition", "VSCode roocode plugin", and lists obsolete directory paths (e.g., `services/openresty`, `api`)

## Decision

We will rewrite the architecture documentation to reflect the current system state, splitting the monolithic flows page into focused pages for implemented features only. Planned-only features will not get dedicated pages until implementation begins — this avoids documentation maintenance burden for things that don't exist yet.

Authentication documentation will be consolidated into a single page (merging `authentication.md` and `flows/aai.md`), since they cover the same topic and currently contradict each other.

CLAUDE.md will be updated to match the rewritten docs.

## Out of Scope

The following are related but intentionally excluded from this ADR:

- Updating the `services/web` README or other per-service READMEs
- Updating docker-compose documentation or environment setup guides
- Updating the monorepo root `package.json` scripts section
- Rewriting the `docs/product/` pages outside of `architecture/`
- Creating a new architecture diagram (the obsolete overview chart was removed in `70d0d1f3`; a replacement diagram should be added separately once the text is stable)
- Adding a "Platform Rationale" section to `overview.md` explaining the positioning and integration rationale for GitLab, Waldur, Kubeflow, MLflow, and Open OnDemand in the stack (should be addressed in a follow-up once the text from this ADR is stable)

## Execution

This work will be carried out in three sequential phases. Each phase has explicit verification steps to confirm correctness before proceeding.

## Plan

### Phase 1: Fix existing pages

_No dependencies. Can begin immediately._

#### 1. Delete `ai-assisted-development.md`

- Delete the file
- Remove from `architecture/_meta.json`

#### 2. Rewrite `overview.md`

- Fix service names to match reality (`services/web`, `shared/gitlab-client`, etc.)
- Replace all `gitlab-secure-proxy` references with `gitlab-root-actor` (the directory `services/gitlab-secure-proxy` no longer exists — the service is now `services/gitlab-root-actor`, Fastify, port 3001 — user lifecycle, impersonation tokens)
- Add `k8s-root-actor` (Fastify, port 3002 — inference/space deployments)
- Add `mlflow-ui` and `waldur-k8s-site-agent` as supporting services
- Remove all Django references and the "transitioning" note
- Update Infrastructure Layer: describe actual docker-compose stack (Valkey, PostgreSQL, OpenResty, Keycloak)
- Update Shared Libs section to reflect `@themelio/gitlab-client`, eslint-config, typescript-config
- Add section on Data Transformation Layer (GitLab → HF transformers, typed GraphQL with codegen)
- Add section on API route structure (resource groups, patterns)
- Keep the tone/depth consistent — high-level architectural rationale, not implementation details

#### 3. Consolidate authentication docs

Merge `authentication.md` and `flows/aai.md` into a single `authentication.md`:

- Replace the Omniauth/Lua proxy description with the actual flow: NextAuth v5, OIDC provider, `gitlab-root-actor` provisioning, Valkey session, middleware token injection
- Include the sequence diagram from `flows/aai.md` (it's accurate)
- Keep the sections on GRNET e-Infra / RCIAM / MyAccessID (still accurate context)
- Update the tokens section to reflect impersonation token lifecycle (create on login, delete on logout, stored in Valkey)
- Update the "local keycloak" section to reflect it's the default dev provider
- Delete `flows/aai.md` after merge (also reflected in step 11 — `flows/_meta.json` update)

#### 4. `gitlab-groups-and-visibility.md`

- Add a note at the top that this design is under review/discussion
- No content changes

#### Verification (Phase 1)

- `grep -r "gitlab-secure-proxy" docs/` returns no results
- `grep -r "gitlab-secure-proxy" CLAUDE.md` returns no results
- `flows/aai.md` no longer exists
- `ai-assisted-development.md` no longer exists

### Phase 2: Split `flows/overview.md` into focused pages (implemented features only)

_Depends on Phase 1 completing (specifically step 3, which deletes `flows/aai.md` and affects the `flows/` directory structure)._

#### 5. `flows/overview.md` → slim down to index page

Becomes a lightweight page linking to the individual flow pages below.

#### 6. `flows/catalog-and-discovery.md` — Implemented

Browsing models/datasets/spaces/repos, search & filtering, resource cards. Mark dataset streaming preview, interactive widgets, and deploy options as planned callouts within the page.

#### 7. `flows/repository-operations.md` — Implemented

Repo CRUD, fork, tree/blob browsing, commit history, settings management. This is the core of what users do today.

#### 8. `flows/deployment.md` — Partially implemented

Inference and space deployment via `k8s-root-actor`. Mark Slurm and Katib as planned callouts within the page.

#### 9. `flows/resource-management.md` — Partially implemented

Space configuration and deployment are real. Mark quota lifecycle and Waldur dashboard as planned callouts within the page.

#### Deferred (not creating pages until implementation begins)

The following are entirely aspirational and will not get dedicated documentation pages yet:

- Compute workflows (Jupyter, Slurm, Katib, SSH)
- MLOps (MLflow/experiment tracking)
- Admin & governance dashboard

#### 10. Remove from `flows/overview.md`

- "User Workflows" section (ML Researcher + ML Engineer persona narratives)
- Navigation Structure / Service Integration Matrix / UX Principles — product spec material, not architecture

#### 11. Update `flows/_meta.json`

- Remove `aai` entry (already merged into `authentication.md` in Phase 1)
- Add entries for new pages: `catalog-and-discovery`, `repository-operations`, `deployment`, `resource-management`

#### 12. Update `architecture/_meta.json`

Remove `ai-assisted-development`, ensure correct ordering.

#### Verification (Phase 2)

- All new flow pages render correctly in the docs site (`pnpm dev` in `docs/`)
- `flows/overview.md` links to all new subpages
- No broken internal links

### Phase 3: Update CLAUDE.md

_Depends on Phases 1 and 2 completing (CLAUDE.md should reflect the final state of the documentation)._

#### 13. Update project CLAUDE.md

- Replace all `gitlab-secure-proxy` references with `gitlab-root-actor`
- Remove "Django/Next.js transition" note (the "Important Notes" section at the bottom)
- Remove "VSCode roocode plugin" and "Task Pipeline Architecture" section
- Update the monorepo structure listing (remove `services/gitlab-secure-proxy`, `services/openresty`, `api`; add `services/gitlab-root-actor`, `services/k8s-root-actor`)
- Update Authentication Flow section to match rewritten docs
- Update Service Architecture section to reflect current services
- Update Individual Service Commands to reference `gitlab-root-actor`

#### Verification (Phase 3)

- `grep -r "gitlab-secure-proxy" .` across the entire repo returns no results
- `grep -r "roocode" .` returns no results
- `grep -r "Django" CLAUDE.md` returns no results

## File Changes Summary

| File | Action |
|------|--------|
| `ai-assisted-development.md` | Delete |
| `overview.md` | Rewrite |
| `authentication.md` | Rewrite (merge with `flows/aai.md`) |
| `flows/aai.md` | Delete (merged into authentication) |
| `gitlab-groups-and-visibility.md` | Add "under discussion" note |
| `flows/overview.md` | Slim down to index page |
| `flows/catalog-and-discovery.md` | New |
| `flows/repository-operations.md` | New |
| `flows/deployment.md` | New |
| `flows/resource-management.md` | New |
| `flows/_meta.json` | Update |
| `architecture/_meta.json` | Update |
| `CLAUDE.md` (project root) | Update |

## Consequences

- **Positive**: Documentation reflects the deployed system, reducing onboarding confusion for new team members and contributors
- **Positive**: Consolidating auth docs eliminates contradictions between `authentication.md` and `flows/aai.md`
- **Positive**: Deferring planned-only pages reduces ongoing maintenance burden
- **Positive**: CLAUDE.md stays in sync, improving AI agent effectiveness for automated tasks
- **Negative**: Some aspirational features lose their documentation — but they were misleading without implementation status markers anyway
- **Negative**: No new architecture diagram is produced by this ADR (the obsolete one was already removed). A replacement diagram should be considered as a follow-up once the text is stable.
- **Risk**: If implementation moves fast, new pages for deferred features will need to be created. This is acceptable — documenting after implementation is the correct order