> For AI agents: the complete documentation index is available at /llms.txt, the full documentation bundle is available at /llms-full.txt. > **Themelio compatibility reference.** This page describes the retained Themelio implementation or its historical design. Check the [parallel-operation guide](/compatibility/index.md) before applying it to the current platform. # User Stories **Version:** 0.2 | **Status:** Under Review | **Date:** 2026-2-20 This document contains the User Stories for Themelio. For complete context including executive vision, scope, and user personas, please refer to the [Core Specifications](/product/requirements/core.md) and the [Introduction to Themelio](/product/introduction.md). ## Epic 1: Authentication, Authorization, and Identity (AAI) **Context:** Centralized gateway for Academic (eduGAIN) and Citizen (eIDAS) login with strict project-based billing and RBAC. ### Story 1.1: Unified Login Gateway - **As a** User (Researcher or Citizen), - **I want** to log in using my existing University or eIDAS credentials, - **So that** I don't have to manage a new set of credentials and my identity is automatically verified. **Acceptance Criteria:** - [x] User can authenticate via [eduGAIN](/terms/edugain.md) (Academic path). - [x] User can authenticate via [eIDAS](/terms/eidas.md) node (Citizen path). - [x] Attributes are auto-ingested from the IdP upon successful login. - [ ] Session is shared transparently across Gitlab, Open OnDemand, and Billing subsystems (SSO). ### Story 1.2: Project Context Switching - **As a** User working on multiple grants, - **I want** to switch my active "Project Context" in the UI, - **So that** my compute and storage usage is billed to the correct grant and I access the correct isolated resources. **Acceptance Criteria:** - [ ] UI provides a selector to switch the active Project. - [ ] Visual indicator shows the currently active Project. - [ ] Switching context updates available quotas and billing ledger target. - [ ] [RBAC](/terms/rbac.md) permissions propagate immediately to [Slurm](/terms/slurm.md)/K8s upon switch. ### Story 1.3: Project Access Management, Waldur, and Manual Administration - **As a** Project Owner, - **I want** to manage user access to my project (invite, revoke, assign co-owners), - **So that** I can collaborate with my team while maintaining security over my allocated resources. **Acceptance Criteria:** - [ ] Owner can invite a user to the project via email or username. - [ ] Owner can revoke a user's access. - [ ] Owner can assign "Co-owner" role to delegate management. - [ ] Owner can view a list of all current members and their roles. - [ ] Invited users should sign an agreement and provide their SSH public keys ### Story 1.4: Waldur - **As a** System Admin, - **I want** to review and approve/reject project applications and modify quotas in Waldur, - **So that** I can ensure resources are allocated to valid research grants and govern the infrastructure. **Acceptance Criteria:** - [ ] Admin interface to list pending project applications. - [ ] Ability to Approve (triggering group creation & metadata assignment) or Reject (with reason) an application, and ask for changes. - [ ] Ability to Approve (triggering group creation & metadata assignment) or Reject (with reason) an application. - [ ] Ability to modify quotas for an active project. - [ ] Ability to force-stop jobs/projects affecting cluster health. ## Epic 2: ML-Optimized Git Repositories **Context:** A Git-based version control system optimized for large ML artifacts (Models/Datasets) with collaboration features. ### Story 2.1: Repository Management - **As a** ML Researcher, - **I want** to create, delete, and clone repositories via HTTP/SSH, - **So that** I can version control my work using standard Git workflows. **Acceptance Criteria:** - [ ] User can upload a new Model, Dataset, or Space repository for the active Project. - [ ] User can download repositories using standard `git clone` (HTTP & SSH). - [ ] UI displays commit history (hashes, authors, timestamps). ### Story 2.2: Large File Support (LFS) - **As a** ML Engineer, - **I want** to push massive binary files (weights, parquet) seamlessly, - **So that** I can version my models and datasets without hitting standard Git size limits. **Acceptance Criteria:** - [ ] Support for git push of files up to TB scale via [LFS](/terms/lfs.md)/Xet protocol. - [ ] Chunk-level deduplication is implemented to optimize storage. - [ ] Standard git clients work without requiring custom plugins for basic push/pull. ### Story 2.3: Collaboration (PRs & Discussions) - **As a** Collaborator, - **I want** to propose changes via Pull Requests and discuss them, - **So that** I can review code and data modifications before they are merged into the main branch. **Acceptance Criteria:** - [ ] User can open a PR from a branch/fork. - [ ] PR supports "Draft" and "Published" states. - [ ] Threaded discussions can be opened on PRs or general topics. - [ ] UI renders visual diffs for supported non-text files (JSON, Images). ## Epic 3: Themelio Catalogue & Discovery **Context:** A directory to publish, discover, and deploy machine learning models. ### Story 3.1: Model Discovery & Filtering - **As a** Researcher, - **I want** to search for models and datasets using tags like Task, Library, and Language, - **So that** I can quickly find a pre-trained model suitable for my specific experiment (e.g., Greek NLP). **Acceptance Criteria:** - [ ] Search interface supports filtering by Task, Library, Language, License. - [ ] Metadata is auto-parsed from `README.md` YAML front matter. - [ ] Search results show metrics: Downloads, Likes, Updated timestamp. - [ ] Private repositories are hidden from public search (ACL respect). ### Story 3.2: Managed Inference Deployment - **As a** Applied ML Engineer, - **I want** to deploy a model repository as a secure inference API endpoint, - **So that** I can integrate the model into my applications without managing the underlying GPU infrastructure. **Acceptance Criteria:** - [ ] One-click deploy from Themelio Catalogue to K8s using the Project resources. - [ ] Auto-selection of container (Triton/vLLM) if not specified. - [ ] API is secured via User Token (OAuth2/PAT). - [ ] API schema follows standard formats (e.g., OpenAI-compatible for LLMs). ### Story 3.3: Interactive Model Widgets - **As a** User, - **I want** to test a model directly in the browser using an interactive widget, - **So that** I can verify if it works for my data (e.g., OCR on a Greek document) before downloading it. **Acceptance Criteria:** - [ ] Widget input UI adapts to pipeline tag (Text, Image, Audio). - [ ] Inference results are rendered visually (JSON pretty print, Image render). - [ ] "Use in API" button generates a curl/python snippet for the current input. ### Story 3.4: Batch Inference with Datasets - **As a** ML Engineer, - **I want** to run batch inference using a trained model on a dataset with a custom script, - **So that** I can process large volumes of data efficiently and implement domain-specific logic. **Acceptance Criteria:** - [ ] User can select a trained model and dataset from the platform when creating a batch inference job - [ ] User provides or selects an inference script that defines how to apply the model to dataset entries - [ ] System validates model compatibility with dataset format and provides clear error messages if incompatible - [ ] Batch job is queued and executed on appropriate compute resources (K8s/Slurm) based on model requirements - [ ] Progress tracking shows number of items processed, remaining, and any errors encountered - [ ] Inference results are automatically saved to a specified output location (new dataset, model repository, or external storage) - [ ] Logs and error reports are captured and accessible for debugging failed inference runs - [ ] Resource usage (GPU/CPU hours, memory) is tracked and attributed to the project quota - [ ] Batch inference job records link the source model, input dataset, and output results - [ ] User can cancel running batch inference jobs with cleanup of partial results ## Epic 4: Dataset Management **Context:** Efficient handling and previewing of large-scale datasets. ### Story 4.1: Dataset Streaming Preview **As a** Data Scientist, **I want** to preview the first 100 rows of a large dataset in the browser without downloading it, **So that** I can verify data quality and schema before committing to a large transfer. **Acceptance Criteria:** - [ ] Viewer streams initial rows via API (no full download). - [ ] Audio and Image columns render players/thumbnails directly in table cells. - [ ] Pagination allows navigation through preview rows. ## Epic 5: Interactive Spaces (App Hosting) **Context:** Hosting for demos and interactive ML applications (Gradio/Streamlit). ### Story 5.1: Space Creation & Auto-Build - **As a** ML Researcher, - **I want** to create a "Space" using Gradio, Streamlit, or Docker, - **So that** I can demonstrate my model to stakeholder or any other user easily. **Acceptance Criteria:** - [ ] User selects SDK (Gradio/Streamlit/Docker) during creation. - [ ] Pushing code triggers auto-build and deployment. - [ ] `requirements.txt` dependencies are installed automatically. ### Story 5.2: CPU Allocation for Spaces - **As a** User, - **I want** to assign specific CPU hardware (or shared CPU) to my Space, - **So that** I can run my demo application. **Acceptance Criteria:** - [ ] Setting a limit for CPU quotas for a consumer user that visit this space. - [ ] Support for "sleep" configuration on idle paid Spaces. - [ ] Shared GPU options for efficient resource usage. ### Story 5.3: Secrets Management - **As a** Developer, - **I want** to securely inject API keys (e.g., OpenAI tokens) into my Space, - **So that** my code works but my secrets aren't exposed in the public repository. **Acceptance Criteria:** - [ ] UI for adding write-only secrets. - [ ] Secrets injected as environment variables at runtime. - [ ] Docker builds support secret mounting via `/run/secrets/`. ## Epic 6: Resource & Quota Management **Context:** Managing the finite hardware resources and budget of projects. ### Story 6.1: Quota Requests - **As a** Project Owner, - **I want** to request initial quotas or extensions through a form, - **So that** I can get the resources I need without out-of-band email communication. **Acceptance Criteria:** - [ ] [Waldur](/terms/waldur.md) interoperability - [ ] Form captures requested GPU hours, Storage quotas, and justification. - [ ] Request routes to Admin Dashboard for approval. - [ ] System handles "Expired" freeze state for a grace period (read-only mode, keep only storage quotas, freeze CPU/GPU resources and jobs). - [ ] Clean up after read-only mode after grace period, TODO: check with Panos Louridas - [ ] After each project is approved, Themelio should call back to [Pharos AAI](/terms/pharos-aai.md) to update the project claims for the specific `extern_uid` so we can propagate this information throughout the whole stack when the user logins again in the future. :::warning When a project expires, should we call back to [Pharos AAI](/terms/pharos-aai.md) to update the group membership and status? Is there an alternative using the expiration of the keycloak group or membership? ::: ### Story 6.2: Budget Monitoring & Alerts - **As a** User, - **I want** to see a dashboard of my resource usage and receive alerts, - **So that** I don't unexpectedly run out of budget or leave expensive jobs running. **Acceptance Criteria:** - [ ] [Waldur](/terms/waldur.md) interoperability - [ ] Dashboard shows Real-time Usage vs Limit (GPU hours, Storage). - [ ] Auto-alerts sent at 50%, 80%, 95% usage thresholds. ## Epic 7: Compute & Training Orchestration **Context:** Access to HPC and Cloud compute for heavy training and exploration. ### Story 7.1: Interactive Notebook Spawning - **As a** Data Scientist, - **I want** to spawn a JupyterLab instance on Kubeflow, - **So that** I can interactively explore data and prototype models. **Acceptance Criteria:** - [ ] User selects hardware [Kubeflow](/terms/kubeflow.md) profile (GPU/CPU). - [ ] User selects for predefined [Kubeflow images](https://www.kubeflow.org/docs/components/notebooks/container-images/#base-images). - [ ] User defines their own [custom images](https://www.kubeflow.org/docs/components/notebooks/container-images/#custom-images) - [ ] `/home` directory persists across stop/start cycles. - [ ] Jupyterlab should include Git integration plugin attached. ### Story 7.2: Batch Job Submission - **As a** ML Engineer, - **I want** to submit batch training jobs to the Slurm/K8s cluster, - **So that** I can train large foundation models that require long-running, massive compute. **Acceptance Criteria:** - [ ] [Open OnDemand](/terms/open-ondemand.md) UI allows defining training [Slurm](/terms/slurm.md) jobs. - [ ] [Slurm](/terms/slurm.md) Job is queued and scheduled based on priority/availability. - [ ] User can cancel a running or queued job. ### Story 7.3: Job Observability - **As a** User, - **I want** to view real-time logs and metrics of my running jobs, - **So that** I can debug failures or monitor training progress. **Acceptance Criteria:** - [ ] Live streaming of `stdout`/`stderr` Open OnDemand. - [ ] Graphs for GPU utilization, Memory, and CPU usage. - [ ] Option to attach a web-terminal to the running node/pod. ### Story 7.4: Model Training Workflow with Datasets - **As a** ML Engineer, - **I want** to train a new model using an existing dataset from the platform, - **So that** I can leverage available data without re-uploading it and track my training experiments systematically. **Acceptance Criteria:** - [ ] User can select a dataset from the platform when configuring a training job - [ ] Dataset is automatically mounted/accessible within the training environment - [ ] Training job configuration captures dataset version, model architecture, and hyperparameters - [ ] Training progress is tracked and can be monitored in real-time - [ ] Training outputs (checkpoints, logs, metrics) are automatically saved to a model repository - [ ] Trained model can be pushed directly to the Model Catalogue upon completion - [ ] Training job records link to both the source dataset and the resulting model ## Epic 8: Experiment Tracking (MLOps) **Context:** Tracking metrics, parameters, and artifacts for reproducibility. ### Story 8.1: MLflow Experiments - **As a** Researcher, - **I want** to log parameters and metrics to a project-specific MLflow instance, - **So that** I can compare different runs and identify the best performing model. **Acceptance Criteria:** - [ ] [MLflow](/terms/mlflow.md) server provisioned per project (or secure multi-tenant). - [ ] UI allows visual comparison of runs. - [ ] Runs are linked to specific Git commits and Dataset versions. ### Story 8.2: Artifact Promotion - **As a** Engineer, - **I want** to push a trained checkpoint from an experiment directly to the MLflow Model Registry, - **So that** I can publish a successful training run without manual download/upload. **Acceptance Criteria:** - [ ] [MLflow](/terms/mlflow.md) UI - [ ] "Push to Registry" action available on Experiment/Run artifact. - [ ] Artifacts are downloadable via UI.
Page source Source snapshot ยท Revision `f928834c8`.