---
title: "ADR-0011: Chat UI and Client Access for Shared LLM Services"
---

# ADR-0011: Chat UI and Client Access for Shared LLM Services

**Status:** Draft

**Date:** 2026-05-22

**Deciders:** Themelio developers (confirmed 2026-06-30)

**Relates to:**
- [ADR-0009 (Shared LLM Inference Backend)](/adr/0009-shared-llm-inference-backend)
- [ADR-0010 (LLM Gateway and Project Token Quotas)](/adr/0010-project-scoped-llm-gateway-token-quotas-api-access)
- [ADR-0003 (Waldur Proposal Workflow)](/adr/0003-waldur-proposal-workflow)
- [ADR-0005 (AAI → Waldur User Synchronization)](/adr/0005-aai-waldur-user-sync)
- [ADR-0020 (LLM Access Membership Source of Truth)](/adr/0020-llm-access-membership-source-of-truth)

## Context

ADR-0009 introduces a shared LLM inference backend based on KServe `LLMInferenceService` and vLLM. ADR-0010 introduces LiteLLM as the governed gateway that maps Waldur projects to LiteLLM Teams, enforces spend budgets, and exposes an OpenAI-compatible API surface.

If Themelio offers only API access, the service works well for SDK clients and coding assistants but is incomplete for many human users. Users also expect a browser-based chat experience: sign in with SSO, use an allowed model, chat with the model, and consume the project budget. The client interaction layer must not bypass or duplicate LiteLLM's budget enforcement.

The non-negotiable architectural rule:

> All user-facing clients — web UI, CLI clients, IDE extensions, SDKs, and agents — **must** call LiteLLM (ADR-0010). Direct access from user clients to vLLM/KServe model endpoints is not allowed.

This means the human web UI must also route through LiteLLM using a project-scoped virtual key. Budget enforcement is performed by LiteLLM regardless of the access path (UI or API).

## Considered Options

### Option A: LibreChat (considered, not adopted)

Deploy LibreChat behind Themelio SSO. LibreChat supports custom OpenAI-compatible endpoints and has stronger multi-provider and agent/MCP workflow features.

**Cons:**
- Additional persistence layer (MongoDB) to operate.
- No public API for automating per-project connection setup.
- Agent/MCP features require additional security review before enabling.

**Decision:** Not adopted at MVP. Remains a candidate if agent/MCP workflows become a hard requirement.

### Option B: Custom Themelio frontend (deferred)

Build a dedicated frontend for shared LLM access.

**Pros:** Full control over quota display, project selection, Themelio branding.
**Cons:** Highest implementation cost; delays service availability.

**Decision:** Long-term option. Not adopted for MVP.

### Option C: Open WebUI (adopted)

Deploy Open WebUI as the browser-based chat UI. Configure it to connect to LiteLLM using per-project virtual keys managed by `llm-access-root-actor`.

**Pros:**
- Self-hosted, full OIDC SSO support without a user count cap.
- Group management API enables automated per-project access control.
- Single Helm chart, straightforward operational model.
- No direct vLLM access — all requests go through LiteLLM.

**Cons:**
- Open WebUI admin edits (groups, connections) are overwritten by root-actor reconciliation.
- Per-project quota display is not surfaced natively in Open WebUI; visible via Waldur usage reports.

**Decision:** Accepted.

### Option D: API-only access (rejected)

Expose only LiteLLM virtual keys with API documentation. No web UI.

**Decision:** Rejected as the production state. Remains valid as an interim milestone before Open WebUI is deployed.

## Why Open WebUI over LibreChat and LiteLLM's built-in UI

| Criterion | LiteLLM built-in UI | LibreChat | Open WebUI (chosen) |
|---|---|---|---|
| OIDC SSO without user cap | No — limited to 5 users in OSS tier | Yes | Yes |
| OpenAI-compatible backend | Native | Yes | Yes |
| Per-project model ACLs | Not applicable (admin UI only) | Limited | Group-scoped model access grants |
| Admin API for automation | Native (used by root actor) | No | Yes (`/openai/config`, `/api/v1/groups/`) |
| Operational simplicity | Single binary | Additional persistence (MongoDB) | Single Helm chart |

LiteLLM's built-in Proxy UI enforces a **5-user limit for SSO/OIDC login** in the open-source tier; API access (virtual keys) has no such restriction. Open WebUI avoids this limit, supports full OIDC SSO with no user cap, and exposes a group management API that `llm-access-root-actor` uses to automate per-project membership without manual admin intervention.

## Decision

Themelio uses **Open WebUI** as the browser-based chat interface and **LiteLLM** as the unified enforcement and routing layer for all client access paths.

- **Open WebUI** provides human-facing SSO chat with per-project model visibility and group-based access control. It connects to LiteLLM via project-scoped virtual keys — it does not call vLLM directly.
- **LiteLLM** enforces budget and throughput limits for every request regardless of whether it originates from Open WebUI or a direct API client.
- **API clients** (SDK, Codex CLI, Claude Code) call LiteLLM directly with a project virtual key.

**See also:** [DD-0011: Implementation details](/dd/0011-human-web-ui-and-agent-client-access)

## Architecture

```mermaid
flowchart TD
    subgraph UI["Human browser"]
        HU[User] -->|OIDC SSO| OW[Open WebUI]
    end

    subgraph API["API / CLI clients"]
        AC[OpenAI SDK / Codex / Claude Code]
    end

    OW -->|per-project virtual key\nOpenAI-compat /v1/| LT[LiteLLM gateway\nADR-0010]
    AC -->|per-project virtual key\nOpenAI-compat /v1/| LT

    LT -->|budget + TPM check| BU[Spend enforcement]
    LT -->|internal alias Service| VL[vLLM\nADR-0009]
    LT -->|spend update| SL[Spend logs → Waldur]

    RA[llm-access-root-actor] -->|upsert connection\n+ model ACLs| OW
    RA -->|create team\n+ virtual key| LT
```

### Human user request path

1. The user authenticates to Open WebUI via OIDC (Keycloak / AAI).
2. Open WebUI resolves the user's group membership and shows project-prefixed models (e.g., `projectid.model-name`).
3. The user selects a model; Open WebUI routes the request to LiteLLM using the per-project virtual key stored in that project's connection entry.
4. LiteLLM checks the team's `max_budget` and optional `tpm_limit`/`rpm_limit`.
5. If within limits, LiteLLM routes to the vLLM backend from ADR-0009.
6. LiteLLM records spend and decrements the team budget.

### API user request path

API users receive the per-project LiteLLM virtual key (via the Waldur offering / ADR-0010 key distribution). They call LiteLLM's OpenAI-compatible endpoint directly. The same budget and model allowlist checks apply.

### Per-project connection provisioning

`llm-access-root-actor` reconciles both the LiteLLM side (team, key) and the Open WebUI side (connection entry, group, model ACLs) in a single idempotent call per Waldur project lifecycle event. See [DD-0011](/dd/0011-human-web-ui-and-agent-client-access) for the connection data model.

## API Surface

LiteLLM's OpenAI-compatible endpoint covers both UI and API access paths:

- `GET /v1/models` — project-filtered model list (filtered by team allowlist)
- `POST /v1/chat/completions` — streaming and non-streaming
- Tool/function calling where the backend model supports it
- Standard OpenAI rate-limit headers
- Anthropic models proxied through the same surface

For Codex CLI clients that require the Responses API (`POST /v1/responses`), a separate `responses-proxy` service translates to Chat Completions. This is outside the LiteLLM gateway layer and is not part of this ADR.

## Consequences

**Positive:**
- Human users get browser-based shared LLM access via OIDC SSO without installing any developer tools.
- API clients, Open WebUI users, and coding tools all consume the same project budget through the same LiteLLM enforcement point.
- Budget enforcement is centralized in LiteLLM regardless of access path; Open WebUI cannot bypass it.
- Per-project model visibility is enforced through Open WebUI group ACLs (group name = `resource_backend_id`).

**Negative:**
- Open WebUI group membership and connection state are externally managed; admin edits in the Open WebUI UI are overwritten on the next reconciliation cycle.
- Project-level quota visibility is available in Waldur and LiteLLM spend logs, but not surfaced in the Open WebUI chat UI.
- Chat history is stored in Open WebUI's SQLite/PVC; retention and deletion policy must be explicitly addressed before production.

**Neutral:**
- File upload, terminals, and pipelines are disabled at deployment (`ENABLE_DIRECT_CONNECTIONS: false`).
- Open WebUI connects to LiteLLM; it does not need direct access to vLLM or the inference namespace.

## Open Questions

~~1. Should the web UI be embedded in Themelio or deployed as a separate service with Themelio branding?~~
**Resolved:** Deployed as a separate service in its own `open-webui` namespace. No Themelio branding customization at MVP.

~~2. How should users choose the active Waldur project in the web UI?~~
**Resolved:** Project selection is implicit via model naming. Each project's models are prefixed `<resource_backend_id>.<model>`. Users see only models accessible to their Open WebUI groups (one group per Waldur project they are a member of).

~~3. Should each project receive a shared API key hidden behind the UI, or should the UI call the gateway using the user's identity?~~
**Resolved:** Shared per-project virtual key stored in a Kubernetes Secret and used by the Open WebUI connection entry. The key is scoped to the LiteLLM Team (project), not the individual user.

~~4. Should conversation history be per-user, per-project, both, or disabled for MVP?~~
**Resolved:** Per-user in Open WebUI's local SQLite persistence (PVC-backed). No cross-project isolation of conversation history at MVP.

~~5. Can Open WebUI be configured to display quota without forking?~~
**Resolved:** No native quota display in Open WebUI. Budget visibility is available via Waldur usage reports and LiteLLM admin spend logs. Not surfaced in the chat UI at MVP.

~~6. Should file upload and RAG features be enabled in MVP?~~
**Resolved:** Disabled at deployment (`ENABLE_DIRECT_CONNECTIONS: false`, `tika.enabled: false`, `terminals.enabled: false`).

~~7. Which external clients are officially supported versus best-effort?~~
**Resolved:** Officially supported: Open WebUI (human UI) + OpenAI SDK clients via LiteLLM API. Best-effort: Codex CLI (via `responses-proxy`), Claude Code (direct LiteLLM virtual key with `ANTHROPIC_BASE_URL` pointing to LiteLLM).

~~8. What model capabilities are required for coding assistant support?~~
**Resolved:** Determined per model via `inference.yml` catalog fields (`enable_auto_tool_choice`, `tool_call_parser`, `max_model_len`).

~~9. How will Themelio document setup for Codex, Claude Code, and generic SDK clients?~~
**Resolved:** Deferred to operations documentation. The mechanism is: set base URL to the LiteLLM external endpoint and use the project virtual key as the API key.

~~10. Should Themelio provide client-side helper scripts for quota status bars?~~
**Resolved:** Not implemented at MVP. LiteLLM's standard rate-limit headers are returned on every response.

## References

- `devops/ansible/roles/open-webui-deploy/` — Open WebUI Helm deployment role
- `devops/ansible/roles/litellm-deploy/` — LiteLLM deployment role (model registration)
- `services/llm-access-root-actor/` — reconciles Open WebUI connections, groups, and model ACLs
- `gitops/manifests/open-webui/` — Kubernetes/ArgoCD manifests for the open-webui namespace
- [Open WebUI documentation](https://docs.openwebui.com/)
- [Open WebUI: SSO / OAuth / OIDC](https://docs.openwebui.com/features/authentication-access/auth/sso/)
- [ADR-0009: Shared LLM Inference Backend](/adr/0009-shared-llm-inference-backend)
- [ADR-0010: LLM Gateway and Project Token Quotas](/adr/0010-project-scoped-llm-gateway-token-quotas-api-access)
- [ADR-0020: LLM Access Membership Source of Truth](/adr/0020-llm-access-membership-source-of-truth)