> For AI agents: the complete documentation index is available at /llms.txt, the full documentation bundle is available at /llms-full.txt. # ADR-0002: Architecture Documentation Update ## Status Proposed ## Context The architecture documentation under `docs/product/architecture/` has fallen out of sync with the codebase. Key problems: - **Wrong service names**: References `themelio-web`, `themelio-web/api`, `themelio-gitlab-client` — actual paths are `services/web`, `services/web/src/app/api/`, `shared/gitlab-client` - **Deprecated services undocumented**: `gitlab-secure-proxy` no longer exists on disk — it was replaced by `services/gitlab-root-actor` (Fastify, port 3001 — user lifecycle, impersonation tokens). `k8s-root-actor` (Fastify, port 3002 — inference/space deployments) is also undocumented - **Authentication page describes a deprecated flow**: GitLab Omniauth + OpenResty Lua proxy is the old approach. Actual auth: NextAuth v5, OIDC provider, `gitlab-root-actor` provisioning, Valkey session, middleware token injection - **Flows page is a monolith**: `flows/overview.md` contains 7+ mermaid charts mixing implemented and aspirational features with no distinction - **Obsolete page**: `ai-assisted-development.md` references Django and VSCode roocode plugin — both gone - **CLAUDE.md is stale**: Still references `gitlab-secure-proxy`, "Django/Next.js transition", "VSCode roocode plugin", and lists obsolete directory paths (e.g., `services/openresty`, `api`) ## Decision We will rewrite the architecture documentation to reflect the current system state, splitting the monolithic flows page into focused pages for implemented features only. Planned-only features will not get dedicated pages until implementation begins — this avoids documentation maintenance burden for things that don't exist yet. Authentication documentation will be consolidated into a single page (merging `authentication.md` and `flows/aai.md`), since they cover the same topic and currently contradict each other. CLAUDE.md will be updated to match the rewritten docs. ## Out of Scope The following are related but intentionally excluded from this ADR: - Updating the `services/web` README or other per-service READMEs - Updating docker-compose documentation or environment setup guides - Updating the monorepo root `package.json` scripts section - Rewriting the `docs/product/` pages outside of `architecture/` - Creating a new architecture diagram (the obsolete overview chart was removed in `70d0d1f3`; a replacement diagram should be added separately once the text is stable) - Adding a "Platform Rationale" section to `overview.md` explaining the positioning and integration rationale for GitLab, Waldur, Kubeflow, MLflow, and Open OnDemand in the stack (should be addressed in a follow-up once the text from this ADR is stable) ## Execution This work will be carried out in three sequential phases. Each phase has explicit verification steps to confirm correctness before proceeding. ## Plan ### Phase 1: Fix existing pages _No dependencies. Can begin immediately._ #### 1. Delete `ai-assisted-development.md` - Delete the file - Remove from `architecture/_meta.json` #### 2. Rewrite `overview.md` - Fix service names to match reality (`services/web`, `shared/gitlab-client`, etc.) - Replace all `gitlab-secure-proxy` references with `gitlab-root-actor` (the directory `services/gitlab-secure-proxy` no longer exists — the service is now `services/gitlab-root-actor`, Fastify, port 3001 — user lifecycle, impersonation tokens) - Add `k8s-root-actor` (Fastify, port 3002 — inference/space deployments) - Add `mlflow-ui` and `waldur-k8s-site-agent` as supporting services - Remove all Django references and the "transitioning" note - Update Infrastructure Layer: describe actual docker-compose stack (Valkey, PostgreSQL, OpenResty, Keycloak) - Update Shared Libs section to reflect `@themelio/gitlab-client`, eslint-config, typescript-config - Add section on Data Transformation Layer (GitLab → HF transformers, typed GraphQL with codegen) - Add section on API route structure (resource groups, patterns) - Keep the tone/depth consistent — high-level architectural rationale, not implementation details #### 3. Consolidate authentication docs Merge `authentication.md` and `flows/aai.md` into a single `authentication.md`: - Replace the Omniauth/Lua proxy description with the actual flow: NextAuth v5, OIDC provider, `gitlab-root-actor` provisioning, Valkey session, middleware token injection - Include the sequence diagram from `flows/aai.md` (it's accurate) - Keep the sections on GRNET e-Infra / RCIAM / MyAccessID (still accurate context) - Update the tokens section to reflect impersonation token lifecycle (create on login, delete on logout, stored in Valkey) - Update the "local keycloak" section to reflect it's the default dev provider - Delete `flows/aai.md` after merge (also reflected in step 11 — `flows/_meta.json` update) #### 4. `gitlab-groups-and-visibility.md` - Add a note at the top that this design is under review/discussion - No content changes #### Verification (Phase 1) - `grep -r "gitlab-secure-proxy" docs/` returns no results - `grep -r "gitlab-secure-proxy" CLAUDE.md` returns no results - `flows/aai.md` no longer exists - `ai-assisted-development.md` no longer exists ### Phase 2: Split `flows/overview.md` into focused pages (implemented features only) _Depends on Phase 1 completing (specifically step 3, which deletes `flows/aai.md` and affects the `flows/` directory structure)._ #### 5. `flows/overview.md` → slim down to index page Becomes a lightweight page linking to the individual flow pages below. #### 6. `flows/catalog-and-discovery.md` — Implemented Browsing models/datasets/spaces/repos, search & filtering, resource cards. Mark dataset streaming preview, interactive widgets, and deploy options as planned callouts within the page. #### 7. `flows/repository-operations.md` — Implemented Repo CRUD, fork, tree/blob browsing, commit history, settings management. This is the core of what users do today. #### 8. `flows/deployment.md` — Partially implemented Inference and space deployment via `k8s-root-actor`. Mark Slurm and Katib as planned callouts within the page. #### 9. `flows/resource-management.md` — Partially implemented Space configuration and deployment are real. Mark quota lifecycle and Waldur dashboard as planned callouts within the page. #### Deferred (not creating pages until implementation begins) The following are entirely aspirational and will not get dedicated documentation pages yet: - Compute workflows (Jupyter, Slurm, Katib, SSH) - MLOps (MLflow/experiment tracking) - Admin & governance dashboard #### 10. Remove from `flows/overview.md` - "User Workflows" section (ML Researcher + ML Engineer persona narratives) - Navigation Structure / Service Integration Matrix / UX Principles — product spec material, not architecture #### 11. Update `flows/_meta.json` - Remove `aai` entry (already merged into `authentication.md` in Phase 1) - Add entries for new pages: `catalog-and-discovery`, `repository-operations`, `deployment`, `resource-management` #### 12. Update `architecture/_meta.json` Remove `ai-assisted-development`, ensure correct ordering. #### Verification (Phase 2) - All new flow pages render correctly in the docs site (`pnpm dev` in `docs/`) - `flows/overview.md` links to all new subpages - No broken internal links ### Phase 3: Update CLAUDE.md _Depends on Phases 1 and 2 completing (CLAUDE.md should reflect the final state of the documentation)._ #### 13. Update project CLAUDE.md - Replace all `gitlab-secure-proxy` references with `gitlab-root-actor` - Remove "Django/Next.js transition" note (the "Important Notes" section at the bottom) - Remove "VSCode roocode plugin" and "Task Pipeline Architecture" section - Update the monorepo structure listing (remove `services/gitlab-secure-proxy`, `services/openresty`, `api`; add `services/gitlab-root-actor`, `services/k8s-root-actor`) - Update Authentication Flow section to match rewritten docs - Update Service Architecture section to reflect current services - Update Individual Service Commands to reference `gitlab-root-actor` #### Verification (Phase 3) - `grep -r "gitlab-secure-proxy" .` across the entire repo returns no results - `grep -r "roocode" .` returns no results - `grep -r "Django" CLAUDE.md` returns no results ## File Changes Summary | File | Action | | --------------------------------- | ----------------------------------- | | `ai-assisted-development.md` | Delete | | `overview.md` | Rewrite | | `authentication.md` | Rewrite (merge with `flows/aai.md`) | | `flows/aai.md` | Delete (merged into authentication) | | `gitlab-groups-and-visibility.md` | Add "under discussion" note | | `flows/overview.md` | Slim down to index page | | `flows/catalog-and-discovery.md` | New | | `flows/repository-operations.md` | New | | `flows/deployment.md` | New | | `flows/resource-management.md` | New | | `flows/_meta.json` | Update | | `architecture/_meta.json` | Update | | `CLAUDE.md` (project root) | Update | ## Consequences - **Positive**: Documentation reflects the deployed system, reducing onboarding confusion for new team members and contributors - **Positive**: Consolidating auth docs eliminates contradictions between `authentication.md` and `flows/aai.md` - **Positive**: Deferring planned-only pages reduces ongoing maintenance burden - **Positive**: CLAUDE.md stays in sync, improving AI agent effectiveness for automated tasks - **Negative**: Some aspirational features lose their documentation — but they were misleading without implementation status markers anyway - **Negative**: No new architecture diagram is produced by this ADR (the obsolete one was already removed). A replacement diagram should be considered as a follow-up once the text is stable. - **Risk**: If implementation moves fast, new pages for deferred features will need to be created. This is acceptable — documenting after implementation is the correct order
Page source Source snapshot · Revision `f928834c8`.