--- title: Overview --- # Architecture Overview Themelio is an open-source ML software stack that gives research teams a single place to store models and datasets, deploy inference endpoints and interactive applications, and manage compute and HPC resources for running model training & finetuning, and other ML workflows — all backed by federated institutional authentication. Under the hood, Themelio wraps GitLab CE as a repository storage and collaboration backend, adds a purpose-built web layer for ML workflows, and orchestrates workflows and deployments on Kubernetes and Slurm. Users never interact with GitLab directly; Themelio handles identity, access control, and API translation so that researchers see a familiar, Hugging Face-style experience. This page covers the system's moving parts and how they fit together. The pages that follow go deeper into [authentication](./authentication), [resource types](../requirements/core#themelio-catalogue), and [networking](./networking). ## System Diagram The diagram below shows the three main layers: the **platform** (application services running on Kubernetes), **Waldur** (resource management and provisioning), and **compute infrastructure** (HPC, GPU clusters, and storage). ```mermaid %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% graph TB %% ── Users ────────────────────────────────────────── User["ML Researcher / Data Scientist"] GitCli["Git Client (HTTPS · SSH)"] %% ── Platform (K8s Cluster) ───────────────────────── Ingress["Istio IngressGateway"] User -->|browser| Ingress GitCli -->|"PAT / SSH key"| Ingress subgraph Platform["Kubernetes Cluster"] PharosAAI["Pharos AAI\n(Keycloak · eduGAIN · eIDAS · OIDC)"] Valkey["Valkey\n(sessions ↔ tokens)"] Web["Themelio Web\n(Next.js)"] GitlabRA["gitlab-root-actor\n(Fastify)"] K8sRA["k8s-root-actor\n(Fastify)"] GitLab["GitLab CE (hidden)"] KServe["KServe"] Kubeflow["Kubeflow"] MLflow["MLflow"] OOD["Open OnDemand"] Grafana["Prometheus · Grafana\nLoki · Mimir · Tempo"] end Ingress --> Web Ingress -->|git smart HTTP| GitLab Web -->|OIDC| PharosAAI Web -->|session| Valkey Web --> GitlabRA Web --> K8sRA Web -->|"GraphQL · REST\n(impersonation token)"| GitLab GitlabRA -->|"REST (root PAT)"| GitLab GitlabRA -->|tokens| Valkey %% ── Waldur ───────────────────────────────────────── subgraph WaldurLayer["Waldur (Resource Management)"] Waldur["Proposals Module\nisolation · access control\nquota mgmt · billing"] WCloud["cloud-site-agent\n(planned)"] WHPC["hpc-site-agent\n(planned)"] Waldur --> WCloud & WHPC end KubeflowRA["kubeflow-root-actor\n(Fastify · planned)"] MLflowRA["mlflow-root-actor\n(Fastify · planned)"] WCloud --> KubeflowRA WCloud --> MLflowRA KubeflowRA -->|profile provisioning| Kubeflow MLflowRA -->|instance provisioning| MLflow WHPC -->|job submission · access provisioning| HPC %% ── Compute ──────────────────────────────────────── subgraph Compute["Compute Land (Daedalus + Cloud)"] K8sGPU["K8s Cluster (GPUs)"] HPC["HPC (Slurm)"] Ceph["Ceph Storage (rook.io)"] end K8sRA --> K8sGPU KServe --> K8sGPU Kubeflow --> K8sGPU HPC --> OOD WHPC -->|infrastructure provisioning| OOD %% ── Offerings ────────────────────────────────────── EuroHPC["EuroHPC"] --> Compute National["National"] --> Compute GRNET["GRNET"] --> Compute StorageOff["Storage Offering"] --> Ceph ``` ## Services Themelio runs as a bundle of cooperating services, each with a clear responsibility boundary. ### Themelio Web The primary application — a Next.js 15+ app that serves both the UI and API routes. It handles everything a user sees and touches: - **UI**: React 19 with server-side rendering. Catalogue pages, file browsers, deployment dashboards, settings. - **API routes**: Organized by resource type (models, datasets, spaces, repos, recipes, upskilling materials). These routes translate between the Themelio API and GitLab's GraphQL and REST APIs. - **Authentication**: NextAuth v5 with [OIDC](../../terms/oidc). Sessions stored in [Valkey](../../terms/valkey). - **Middleware**: Intercepts every authenticated request, retrieves the user's GitLab [impersonation token](../../terms/impersonation-token) from [Valkey](../../terms/valkey), and injects it as a `PRIVATE-TOKEN` header. Downstream code never handles raw credentials. The API routes live inside the same Next.js process, avoiding cross-service latency for user-facing operations. ### Cloud Site Agent _(planned)_ A Waldur site agent plugin that provisions cloud-based ML platform resources when a project proposal is accepted: - **Kubeflow resources**: Delegates profile and namespace provisioning to the Kubeflow Root Actor. - **MLflow resources**: Delegates tenant and workspace provisioning to the MLflow Root Actor. - **Order lifecycle**: Handles order processing, membership synchronization, and usage reporting via the Waldur site agent framework. The cloud site agent does not hold privileged credentials itself — it delegates all privileged operations to the Kubeflow and MLflow root actors, maintaining the same separation of concerns used across the platform. ### HPC Site Agent _(planned)_ A Waldur site agent plugin that provisions HPC infrastructure to give users access to Slurm-based compute resources: - **Slurm access**: Provisions user accounts and project allocations on HPC clusters. - **Open OnDemand**: Configures interactive compute sessions (Jupyter, RStudio, virtual desktops) for project members. - **Storage mounts**: Sets up shared filesystem access between HPC nodes and project storage. - **Order lifecycle**: Handles order processing, membership synchronization, and usage reporting via the Waldur site agent framework. ### GitLab Root Actor A Fastify service that holds the GitLab root PAT and manages user lifecycle: - **User provisioning**: Creates GitLab accounts and group structures on first login. - **Impersonation tokens**: Creates tokens on sign-in, revokes them on sign-out. - **Personal access tokens**: Issues scoped tokens for HTTPS git operations. :::warning GitLab root PAT should be regularly rotated and handled as one of the most sensitive security concerns. ::: This service exists because the web layer should never hold root credentials. It acts as a narrow, auditable gateway to GitLab admin operations. ### K8s Root Actor A Fastify service that holds cluster-admin credentials and manages Kubernetes deployments: - **Inference endpoints**: Deploys models behind HTTP endpoints with Istio routing. - **Spaces**: Deploys interactive applications (Gradio, Streamlit, Docker, static sites). - **Custom actions**: Executes repository-defined workflows (`actions.yaml`) as Kubernetes deployments or Slurm jobs. - **Lifecycle management**: Tracks deployment state, collects logs, manages cleanup. Like the GitLab Root Actor, this service isolates privileged operations behind a purpose-built API. ### Kubeflow Root Actor _(planned)_ A Fastify service that holds Kubeflow admin credentials and manages Kubeflow resources on behalf of the cloud site agent: - **Profile provisioning**: Creates Kubeflow profiles and namespaces for project teams. - **RBAC management**: Configures role bindings and access policies for notebooks and pipelines. - **GPU resource allocation**: Manages GPU quota assignments for training workloads. This service isolates Kubeflow admin operations behind a narrow API, following the same root actor pattern as the GitLab and K8s root actors. It is invoked by the cloud site agent during order processing. ### MLflow Root Actor _(planned)_ A Fastify service that holds MLflow admin credentials and manages MLflow resources on behalf of the cloud site agent: - **Tenant provisioning**: Creates MLflow tenants and workspaces for project teams. - **Permission management**: Configures access controls for experiments and model registries. - **Usage reporting**: Tracks resource consumption for billing and quota enforcement. Like the other root actors, this service isolates MLflow admin operations behind a purpose-built API. It is invoked by the cloud site agent during order processing. ### Supporting Services | Service | Role | | ----------------- | -------------------------------------------------------------------------------------------------- | | **[Valkey](../../terms/valkey)** | Redis-compatible store for session data and [impersonation token](../../terms/impersonation-token) mapping | | **GitLab CE** | Git backend, repository storage, CI/CD. Not exposed to users directly | | **[Waldur](../../terms/waldur)** | Resource management platform — proposals, quota enforcement, billing, and site agent orchestration | | **[Kubeflow](../../terms/kubeflow)** | ML pipelines, notebook servers, and distributed training workloads | | **[MLflow](../../terms/mlflow)** | Experiment tracking, model registry, and artifact storage | | **[Open OnDemand](../../terms/open-ondemand)** | HPC portal for interactive compute sessions (Jupyter, RStudio, virtual desktops) | ## Infrastructure ### Kubernetes Kubernetes is the deployment target across all environments. Services run as pods with Kustomize overlays per environment (local, dev, staging, production). - **Local development**: Minikube with host filesystem mounts for hot-reload, or Docker Compose for a lighter setup - **Production**: AWS infrastructure provisioned via Terraform (VPC, EC2, ECR, S3) - **Configuration management**: Ansible (41 roles covering identity, storage, networking, and ML platform components) ### Istio Istio provides the service mesh: - **Ingress**: Separate gateways for web traffic and deployed user workloads (spaces, inferences) - **Routing**: VirtualService-based routing splits git smart HTTP operations from web requests - **Security**: Mutual TLS between services in the mesh See [Networking](./networking) for details on how routing works. ### GitOps ArgoCD manages all deployments with automated sync, pruning, and self-healing. Each environment has a root ArgoCD Application that sources its configuration from a specific Kustomize overlay. Sync waves control deployment order — infrastructure resources deploy before application services. ### Storage and Data | Component | Purpose | | -------------- | ------------------------------------------------------------------------ | | **PostgreSQL** | GitLab persistence | | **MinIO** | S3-compatible object storage for Git LFS (model weights, large datasets) | | **Ceph** | Distributed storage for production workloads (via rook.io) | ## Shared Libraries | Package | Purpose | | ------------------------------ | ---------------------------------------------- | | `@themelio/gitlab-client` | Typed GitLab API client shared across services | | `@workspace/eslint-config` | Shared linting rules | | `@workspace/typescript-config` | Shared TypeScript configuration | ## Key Design Decisions | Decision | Rationale | | ------------------------------------------ | ---------------------------------------------------------------------------------------------------- | | GitLab CE as the storage backend | Mature git API, built-in CI/CD, avoids building repository management from scratch | | Impersonation tokens over OAuth delegation | GitLab CE lacks fine-grained OAuth scopes; impersonation provides full API access scoped to the user | | Separate root actor services | Isolates privileged credentials behind narrow, auditable APIs | | Valkey over Redis | Drop-in compatible with open-source governance | | Istio over plain ingress | Smart routing (git vs. web), mTLS, observability hooks | | ArgoCD + Kustomize over Helm | Declarative GitOps with environment overlays, less templating complexity | | Next.js 15 for the web layer | Full-stack development velocity, SSR for data-heavy dashboards, unified deployment | | pnpm workspaces + Turborepo | Dependency management and build orchestration across the monorepo with intelligent caching |