#ADR-0002: Architecture Documentation Update

#Status

Proposed

#Context

The architecture documentation under docs/product/architecture/ has fallen out of sync with the codebase. Key problems:

• Wrong service names: References themelio-web, themelio-web/api, themelio-gitlab-client — actual paths are services/web, services/web/src/app/api/, shared/gitlab-client
• Deprecated services undocumented: gitlab-secure-proxy no longer exists on disk — it was replaced by services/gitlab-root-actor (Fastify, port 3001 — user lifecycle, impersonation tokens). k8s-root-actor (Fastify, port 3002 — inference/space deployments) is also undocumented
• Authentication page describes a deprecated flow: GitLab Omniauth + OpenResty Lua proxy is the old approach. Actual auth: NextAuth v5, OIDC provider, gitlab-root-actor provisioning, Valkey session, middleware token injection
• Flows page is a monolith: flows/overview.md contains 7+ mermaid charts mixing implemented and aspirational features with no distinction
• Obsolete page: ai-assisted-development.md references Django and VSCode roocode plugin — both gone
• CLAUDE.md is stale: Still references gitlab-secure-proxy, "Django/Next.js transition", "VSCode roocode plugin", and lists obsolete directory paths (e.g., services/openresty, api)

#Decision

We will rewrite the architecture documentation to reflect the current system state, splitting the monolithic flows page into focused pages for implemented features only. Planned-only features will not get dedicated pages until implementation begins — this avoids documentation maintenance burden for things that don't exist yet.

Authentication documentation will be consolidated into a single page (merging authentication.md and flows/aai.md), since they cover the same topic and currently contradict each other.

CLAUDE.md will be updated to match the rewritten docs.

#Out of Scope

The following are related but intentionally excluded from this ADR:

• Updating the services/web README or other per-service READMEs
• Updating docker-compose documentation or environment setup guides
• Updating the monorepo root package.json scripts section
• Rewriting the docs/product/ pages outside of architecture/
• Creating a new architecture diagram (the obsolete overview chart was removed in 70d0d1f3; a replacement diagram should be added separately once the text is stable)
• Adding a "Platform Rationale" section to overview.md explaining the positioning and integration rationale for GitLab, Waldur, Kubeflow, MLflow, and Open OnDemand in the stack (should be addressed in a follow-up once the text from this ADR is stable)

#Execution

This work will be carried out in three sequential phases. Each phase has explicit verification steps to confirm correctness before proceeding.

#Plan

#Phase 1: Fix existing pages

No dependencies. Can begin immediately.

#1. Delete ai-assisted-development.md

• Delete the file
• Remove from architecture/_meta.json

#2. Rewrite overview.md

• Fix service names to match reality (services/web, shared/gitlab-client, etc.)
• Replace all gitlab-secure-proxy references with gitlab-root-actor (the directory services/gitlab-secure-proxy no longer exists — the service is now services/gitlab-root-actor, Fastify, port 3001 — user lifecycle, impersonation tokens)
• Add k8s-root-actor (Fastify, port 3002 — inference/space deployments)
• Add mlflow-ui and waldur-k8s-site-agent as supporting services
• Remove all Django references and the "transitioning" note
• Update Infrastructure Layer: describe actual docker-compose stack (Valkey, PostgreSQL, OpenResty, Keycloak)
• Update Shared Libs section to reflect @themelio/gitlab-client, eslint-config, typescript-config
• Add section on Data Transformation Layer (GitLab → HF transformers, typed GraphQL with codegen)
• Add section on API route structure (resource groups, patterns)
• Keep the tone/depth consistent — high-level architectural rationale, not implementation details

#3. Consolidate authentication docs

Merge authentication.md and flows/aai.md into a single authentication.md:

• Replace the Omniauth/Lua proxy description with the actual flow: NextAuth v5, OIDC provider, gitlab-root-actor provisioning, Valkey session, middleware token injection
• Include the sequence diagram from flows/aai.md (it's accurate)
• Keep the sections on GRNET e-Infra / RCIAM / MyAccessID (still accurate context)
• Update the tokens section to reflect impersonation token lifecycle (create on login, delete on logout, stored in Valkey)
• Update the "local keycloak" section to reflect it's the default dev provider
• Delete flows/aai.md after merge (also reflected in step 11 — flows/_meta.json update)

#4. gitlab-groups-and-visibility.md

• Add a note at the top that this design is under review/discussion
• No content changes

#Verification (Phase 1)

• grep -r "gitlab-secure-proxy" docs/ returns no results
• grep -r "gitlab-secure-proxy" CLAUDE.md returns no results
• flows/aai.md no longer exists
• ai-assisted-development.md no longer exists

#Phase 2: Split flows/overview.md into focused pages (implemented features only)

Depends on Phase 1 completing (specifically step 3, which deletes flows/aai.md and affects the flows/ directory structure).

#5. flows/overview.md → slim down to index page

Becomes a lightweight page linking to the individual flow pages below.

#6. flows/catalog-and-discovery.md — Implemented

Browsing models/datasets/spaces/repos, search & filtering, resource cards. Mark dataset streaming preview, interactive widgets, and deploy options as planned callouts within the page.

#7. flows/repository-operations.md — Implemented

Repo CRUD, fork, tree/blob browsing, commit history, settings management. This is the core of what users do today.

#8. flows/deployment.md — Partially implemented

Inference and space deployment via k8s-root-actor. Mark Slurm and Katib as planned callouts within the page.

#9. flows/resource-management.md — Partially implemented

Space configuration and deployment are real. Mark quota lifecycle and Waldur dashboard as planned callouts within the page.

#Deferred (not creating pages until implementation begins)

The following are entirely aspirational and will not get dedicated documentation pages yet:

• Compute workflows (Jupyter, Slurm, Katib, SSH)
• MLOps (MLflow/experiment tracking)
• Admin & governance dashboard

#10. Remove from flows/overview.md

• "User Workflows" section (ML Researcher + ML Engineer persona narratives)
• Navigation Structure / Service Integration Matrix / UX Principles — product spec material, not architecture

#11. Update flows/_meta.json

• Remove aai entry (already merged into authentication.md in Phase 1)
• Add entries for new pages: catalog-and-discovery, repository-operations, deployment, resource-management

#12. Update architecture/_meta.json

Remove ai-assisted-development, ensure correct ordering.

#Verification (Phase 2)

• All new flow pages render correctly in the docs site (pnpm dev in docs/)
• flows/overview.md links to all new subpages
• No broken internal links

#Phase 3: Update CLAUDE.md

Depends on Phases 1 and 2 completing (CLAUDE.md should reflect the final state of the documentation).

#13. Update project CLAUDE.md

• Replace all gitlab-secure-proxy references with gitlab-root-actor
• Remove "Django/Next.js transition" note (the "Important Notes" section at the bottom)
• Remove "VSCode roocode plugin" and "Task Pipeline Architecture" section
• Update the monorepo structure listing (remove services/gitlab-secure-proxy, services/openresty, api; add services/gitlab-root-actor, services/k8s-root-actor)
• Update Authentication Flow section to match rewritten docs
• Update Service Architecture section to reflect current services
• Update Individual Service Commands to reference gitlab-root-actor

#Verification (Phase 3)

• grep -r "gitlab-secure-proxy" . across the entire repo returns no results
• grep -r "roocode" . returns no results
• grep -r "Django" CLAUDE.md returns no results

#File Changes Summary

#Consequences

• Positive: Documentation reflects the deployed system, reducing onboarding confusion for new team members and contributors
• Positive: Consolidating auth docs eliminates contradictions between authentication.md and flows/aai.md
• Positive: Deferring planned-only pages reduces ongoing maintenance burden
• Positive: CLAUDE.md stays in sync, improving AI agent effectiveness for automated tasks
• Negative: Some aspirational features lose their documentation — but they were misleading without implementation status markers anyway
• Negative: No new architecture diagram is produced by this ADR (the obsolete one was already removed). A replacement diagram should be considered as a follow-up once the text is stable.
• Risk: If implementation moves fast, new pages for deferred features will need to be created. This is acceptable — documenting after implementation is the correct order

Page source

Source snapshot · Revision f928834c8.