For AI agents: the complete documentation index is available at /llms.txt, the full documentation bundle is available at /llms-full.txt, and this page is available as Markdown at /reference/docs/dd/0044-llm-stack-service.md.

DD-0044 — The shared LLM stack as the llm service

Related ADR: 0044. Status: draft for review. Date: 20 October 2026.

Scope and responsibilities

This is infrastructure configuration under decision 0040. There is no new business context, aggregate, command, event or subscription.

OwnerResponsibility
Ansible bootstrapThe llm namespace and its Pod Security labels, the services project's destination, the controller's Role, the services cluster binding
Argo CDThe llm chart: generated Secrets, LiteLLM, Open WebUI, the dev-box model and the namespace's NetworkPolicies

Application

gitops/apps/base/llm.yaml installs gitops/manifests/llm/chart on wave 3 with base/values.yaml, pins.yaml and the environment's overlays/<env>/values.yaml. Sync options match GitLab and Waldur.

Chart

ObjectNotes
llm-secrets hook (wave -3)Creates llm-litellm and llm-open-webui once, with only values this namespace generates; the Role it runs under can only read and create Secrets
llm-webui-key hook (PostSync)Registers the chat's gateway key (llm-open-webui/openai-api-key) as an inference-only virtual key, inside the gateway pod; its account llm-gateway-keys may read that one Secret and exec in the gateway pod
llm-postgresql ServiceAn ExternalName to the shared PostgreSQL in pharos-infra; pharos-infra creates the litellm and open_webui roles and databases and their Secret (decision 0048)
litellmDeployment and Service on port 4000; routes from llm.models, written to a ConfigMap whose checksum restarts the pod; a route changes only through GitOps. The same ConfigMap carries the tool_order callback, which the gateway loads only when the catalogue flags a model with strictToolOrder: that model's Responses history gets each tool result next to its call, whatever order the client replayed
llm-open-webui-settings hook (PostSync, wave 1)Reconciles the admin settings a site opts into through Open WebUI's own routes, after the chat's gateway key exists: web search, the embedding model and function calling. It reads each form, merges its own fields and writes only what changed, and writes the embedding key on every sync. A site that names no settings renders no hook
open-webuiDeployment and Service on port 80; its own database, a 2 Gi volume for uploads, the gateway as its only model connection, reached with its own virtual key
llm-cpu-modelA dev box's llama.cpp server for qwen2.5-0.5b-cpu; weights download on first start into a scratch volume on the node's disk
NetworkPoliciesDefault-deny ingress; the chat may reach the gateway and only the gateway may reach the model; egress is not restricted

Web search is a per-chat choice and a site opts into it with llm.openWebui.settings.webSearch. The query leaves the cluster for the search engine the setting names (DuckDuckGo by default), which the platform holds no key for and cannot audit; the retrieved page is then read by the same chat. A member who does not turn it on for a chat sends nothing to an external engine.

Images

gitops/manifests/llm/pins.yaml is the one place the images are written, each by digest: LiteLLM v1.96.0, Open WebUI 0.11.0, the kubectl image Waldur uses, and llama.cpp's server image. The check fails if the chart renders another image or one without a digest.

Access

No route is published. To reach the stack from a workstation:

kubectl --kubeconfig <target kubeconfig> -n llm port-forward svc/litellm 4000
kubectl --kubeconfig <target kubeconfig> -n llm port-forward svc/open-webui 8080:80

The master key and the administrator password are in the llm-litellm and llm-open-webui Secrets.

Not covered

Routes and sign-in through the AAI, KServe and cert-manager, the shared CloudNativePG cluster, and the Metallo reconciler's own credentials.

Page source

Source snapshot · Revision f928834c8.