# GPU serving qualification on staging This is the staging procedure for priority 5 of #1123, within the agreed [review checklist](review-demo-plan.md). Record actual Site capacity before each rehearsal. Source tests and simulated native observations do not qualify hardware, member access, Project networking or a deployed model. ## Deployment prerequisites Deploy one reviewed platform revision and its matching Site Runtime images. Record the delivered Kubernetes Runtime image by immutable digest. For Slurm, build and install the corresponding release through the [supported Site installer](../../devops/hpc/site/README.md), including the Site's GPU/GRES mapping and wall-time limit when GPU training is selected. A platform deployment alone does not upgrade the external Slurm Runtime. Use an owned Project with an existing, ready allocation of the chosen GPU class. The [accelerator catalogue](../../gitops/manifests/gpu-acceleration/classes.yaml) maps that class to its Kubernetes resource and vLLM serving runtime. Check actual Site capacity when preparing the allocation. Activate the [Project-private gateway and policies](../../gitops/manifests/pharos-site-runtime/serving/README.md). The member opens the application on its own Run origin. Gradio calls the model Service within the same canonical Project and Site. Relaunch pre-upgrade services to receive their declared `pharos-http` port name. The platform application image contains a locked offline wheelhouse for Gradio and Streamlit. An application pre-command such as `python -m pip install gradio` uses that wheelhouse. Additional packages need their own reviewed image or package access policy; these checks do not open general internet egress. ## Prepare and launch 1. Configure `gpuClass` in `demo.json` with the allocated class, for example `gh200-mig-1g.18gb`, and configure `inferenceRepository`. M18-11 checks that copying this class puts the exact value on the clipboard. The legacy `gpuLabel` configuration alias must agree when supplied. 2. Commit the real model and serving action, using the generated inference endpoint's port `8080`. M18-28 launches it through the UI, correlates the new Run and records its latest Attempt. It checks Healthy state, the Kubernetes environment, saved GPU class, member entry URL and authorised vLLM logs. Its teardown stops this deployment. 3. Prepare a separate retained serving fixture through the same owned repository's action. Record that Run and Attempt, its native InferenceService UID, namespace, quota UID and delivered Runtime image in `M18_GPU_TARGET`. This file explicitly authorises M18-30 to stop that exact Run. 4. Commit [the Gradio application](fixtures/gradio/app.py) to an owned Space. Its action installs Gradio from the wheelhouse and runs `python app.py`. Supply `MODEL_RUN_ID`, `MODEL_NAME` and `MODEL_BASE_URL` as literal environment values. The base is the owned inference Service's Project-local URL ending in `/v1`, for example `http://attempt--access..svc.cluster.local:8080/v1`. Configure `chatSpaceURL` with that Gradio Run's **Open** URL, starting at `https:///auth/run-services//open`. Record the owned application repository, Run and pinned source commit in `M18_GRADIO_TARGET`. The driver verifies the maintained source, follows the authorised redirect and uses the application directly. 5. Run M18-29 and M18-30 in that order. The release scenario is destructive only to the explicitly owned deployment. It preserves the Project allocation and Run history. A later rehearsal needs a newly launched deployment and a new ownership record. Keep the normal acceptance target, credentials and ownership records in ignored private files. The [provider fixture guide](provider-fixtures.md#gpu-model-and-gradio-m18-2930) defines both target schemas, mode-0600 permissions and the native observer's read and fixed NVIDIA-query access. Use actual owned identities and delivered image digests. ```sh ACCEPTANCE_CONFIG="$PWD/.local/acceptance/staging/target.json" \ M18_CONFIG="$PWD/.local/acceptance/staging/demo.json" \ M18_GPU_TARGET="$PWD/.local/acceptance/staging/gpu.json" \ M18_GRADIO_TARGET="$PWD/.local/acceptance/staging/gradio.json" \ M18_REPORT_DIR="$PWD/.local/acceptance/staging/serving-release-1" \ pnpm acceptance:demo:run --grep 'M18-29|M18-30' ``` Supply the target's normal HTTPS trust configuration from the [acceptance guide](../README.md). ## What constitutes a pass M18-29 checks a healthy owned GPU Run, its native GPU request, an actual NVIDIA device query and its quota usage before opening the exact Gradio application Run. Its pinned source must match the maintained fixture. The displayed model Run and endpoint must match the owned inference Attempt and namespace, then the application must return one completed non-error answer. Incremental streaming is outside the final demo criterion. M18-30 opens the exact owned deployment, confirms its Project, saved Attempt and GPU demand, then uses its UI Stop control and checks the correlated command. Success requires the native service, endpoint and Pods to disappear, all observed DRA allocations to be released, and measured quota usage to decrease by at least the requested count. Quota limits and the Project allocation remain unchanged, the Site must report the measured usage and the Project page must display it. The terminated Run must still appear in repository history. M18-20 performs real Slurm model training using the [maintained training example](training-example.py). It checks the newly submitted Run, saved epochs, pinned source and learned model output. CPU Slurm is sufficient for this separate core check. Retain all original BLOCKED results in historical reports. Record the platform revision, Runtime image, native observations and fresh scenario outcomes for each staging rehearsal. Concurrent GPU use can prevent a conclusive quota decrease; use an exclusive test allocation rather than changing its limits.