# Platform model serving

KServe is an optional dev-box profile and a default dependency of distributed
delivery, including `icarus-cloud` staging. Each selected installation includes
one `kserve` Application. Foundation-only environments install it when application
delivery is enabled. Production activation still follows its environment policy.

The vendored KServe 0.19.0 and LeaderWorkerSet 0.6.0 charts run in the
bootstrap-owned `kserve` namespace. `pins.yaml` records chart checksums, the
duplicate-CRD removal from the upstream bundle, and controller image digests.
The controller images support AMD64 and ARM64. Standard mode uses Kubernetes
Deployments. Knative and the privileged LocalModel agent are disabled. Runtime
inference stays private; KServe does not create public ingress routes.
The hostname template puts the attempt and Project namespace in separate DNS
labels, so canonical IDs fit Kubernetes' label-length limit.

## Installation and updates

Enable model serving with `make deploy TARGET=dev-NAME PROFILES=kserve`, or select
KServe on the Pharos DX Dev box page. Combine optional services with
`PROFILES=zammad,kserve`. Deploy saves the selection; subsequent deploys and
`make reconcile TARGET=dev-NAME` reuse it. Dev boxes without this profile omit
the controller, CPU image build and serving prerequisites. They can still run
Jobs and application services; Hugging Face inference requires KServe.

Distributed sites use `make deploy ENV=NAME` after their foundation has been
qualified and always include KServe. Delivery refreshes bootstrap grants,
namespaces and Argo's cache before applying the Application.

An existing platform-owned KServe installation, including the earlier default
installation, must be retained with `PROFILES=kserve`. Its cluster-wide APIs and
Project workloads outlive a service namespace. Ordinary profile removal therefore
refuses to uninstall KServe, and dev-box reset preserves its Application and
namespace. Retiring it requires a separate reviewed uninstall after all dependent
workloads have stopped. Shared cert-manager and Gateway API installations remain
under their existing owners.

The bootstrap ensures cert-manager and Gateway API first. A complete, ready
external installation is preserved. Fresh sites receive pinned cert-manager
1.19.2 and Gateway API 1.4.1. Interrupted installations resume only through their
recorded ownership. An incomplete or incompatible external installation stops
delivery and must be repaired through its existing owner. Existing KServe
installations require their own reviewed ownership transition before adoption.

## Architecture-specific CPU serving

`pharos-huggingface-cpu` is a ClusterServingRuntime whose image is built through
the platform image catalogue for the target architecture and delivered by digest.
The upstream Hugging Face CPU image lacks an ARM64 manifest. The platform image
uses the upstream 0.19.0 server with frozen CPU dependencies and the Transformers
backend. Its lock file resolves for Linux AMD64 and ARM64.

New CPU inference attempts explicitly select this runtime. Already persisted
attempts retain their recorded runtime and deployment mode during recovery. The
batch image's Git LFS download also follows the target architecture for repository
preparation.

## GPU serving

GPU requests select the `serving_runtime` declared by their accelerator class in
[the shared map](../gpu-acceleration/classes.yaml). The installed
[GPU runtimes](gpu-runtime.yaml) pin the vLLM images for GH200 and L40 classes by
digest. The Runtime supplies the prepared local model, vLLM arguments and its
`/health` readiness probe. Admission permits only the installed CPU and GPU
runtime names. User workloads receive no platform administrator credential.

Qualify the selected image, NVIDIA driver, GPU class and model on the target Site.
The [staging procedure](../../../acceptance/demo/gpu-staging.md) checks actual
allocation, a model response and measured release. Image availability and source
fixtures alone do not establish GPU compatibility.

Installing KServe provides the serving controllers and runtimes. It does not add
the optional model catalogue, model downloads or gateway offerings from the
separate LLM inference work.

## Verification

`pnpm verify` checks default and selected dev-box compositions, distributed
inclusion, chart checksums, immutable
images, restricted Pod Security, scoped controller grants, bootstrap ownership
and Runtime recovery. The native services and pipeline lanes reuse a ready
managed KServe installation and leave its resources out of fixture cleanup.
Their disposable project workloads retain their existing cleanup rules.