Themelio compatibility reference. This page describes the retained Themelio implementation or its historical design. Check the parallel-operation guide before applying it to the current platform.

#User Stories

Version: 0.2 | Status: Under Review | Date: 2026-2-20

This document contains the User Stories for Themelio. For complete context including executive vision, scope, and user personas, please refer to the Core Specifications and the Introduction to Themelio.

#Epic 1: Authentication, Authorization, and Identity (AAI)

Context: Centralized gateway for Academic (eduGAIN) and Citizen (eIDAS) login with strict project-based billing and RBAC.

#Story 1.1: Unified Login Gateway

• As a User (Researcher or Citizen),
• I want to log in using my existing University or eIDAS credentials,
• So that I don't have to manage a new set of credentials and my identity is automatically verified.

Acceptance Criteria:

• User can authenticate via eduGAIN (Academic path).
• User can authenticate via eIDAS node (Citizen path).
• Attributes are auto-ingested from the IdP upon successful login.
• Session is shared transparently across Gitlab, Open OnDemand, and Billing subsystems (SSO).

#Story 1.2: Project Context Switching

• As a User working on multiple grants,
• I want to switch my active "Project Context" in the UI,
• So that my compute and storage usage is billed to the correct grant and I access the correct isolated resources.

Acceptance Criteria:

• UI provides a selector to switch the active Project.
• Visual indicator shows the currently active Project.
• Switching context updates available quotas and billing ledger target.
• RBAC permissions propagate immediately to Slurm/K8s upon switch.

#Story 1.3: Project Access Management, Waldur, and Manual Administration

• As a Project Owner,
• I want to manage user access to my project (invite, revoke, assign co-owners),
• So that I can collaborate with my team while maintaining security over my allocated resources.

Acceptance Criteria:

• Owner can invite a user to the project via email or username.
• Owner can revoke a user's access.
• Owner can assign "Co-owner" role to delegate management.
• Owner can view a list of all current members and their roles.
• Invited users should sign an agreement and provide their SSH public keys

#Story 1.4: Waldur

• As a System Admin,
• I want to review and approve/reject project applications and modify quotas in Waldur,
• So that I can ensure resources are allocated to valid research grants and govern the infrastructure.

Acceptance Criteria:

• Admin interface to list pending project applications.
• Ability to Approve (triggering group creation & metadata assignment) or Reject (with reason) an application, and ask for changes.
• Ability to Approve (triggering group creation & metadata assignment) or Reject (with reason) an application.
• Ability to modify quotas for an active project.
• Ability to force-stop jobs/projects affecting cluster health.

#Epic 2: ML-Optimized Git Repositories

Context: A Git-based version control system optimized for large ML artifacts (Models/Datasets) with collaboration features.

#Story 2.1: Repository Management

• As a ML Researcher,
• I want to create, delete, and clone repositories via HTTP/SSH,
• So that I can version control my work using standard Git workflows.

Acceptance Criteria:

• User can upload a new Model, Dataset, or Space repository for the active Project.
• User can download repositories using standard git clone (HTTP & SSH).
• UI displays commit history (hashes, authors, timestamps).

#Story 2.2: Large File Support (LFS)

• As a ML Engineer,
• I want to push massive binary files (weights, parquet) seamlessly,
• So that I can version my models and datasets without hitting standard Git size limits.

Acceptance Criteria:

• Support for git push of files up to TB scale via LFS/Xet protocol.
• Chunk-level deduplication is implemented to optimize storage.
• Standard git clients work without requiring custom plugins for basic push/pull.

#Story 2.3: Collaboration (PRs & Discussions)

• As a Collaborator,
• I want to propose changes via Pull Requests and discuss them,
• So that I can review code and data modifications before they are merged into the main branch.

Acceptance Criteria:

• User can open a PR from a branch/fork.
• PR supports "Draft" and "Published" states.
• Threaded discussions can be opened on PRs or general topics.
• UI renders visual diffs for supported non-text files (JSON, Images).

#Epic 3: Themelio Catalogue & Discovery

Context: A directory to publish, discover, and deploy machine learning models.

#Story 3.1: Model Discovery & Filtering

• As a Researcher,
• I want to search for models and datasets using tags like Task, Library, and Language,
• So that I can quickly find a pre-trained model suitable for my specific experiment (e.g., Greek NLP).

Acceptance Criteria:

• Search interface supports filtering by Task, Library, Language, License.
• Metadata is auto-parsed from README.md YAML front matter.
• Search results show metrics: Downloads, Likes, Updated timestamp.
• Private repositories are hidden from public search (ACL respect).

#Story 3.2: Managed Inference Deployment

• As a Applied ML Engineer,
• I want to deploy a model repository as a secure inference API endpoint,
• So that I can integrate the model into my applications without managing the underlying GPU infrastructure.

Acceptance Criteria:

• One-click deploy from Themelio Catalogue to K8s using the Project resources.
• Auto-selection of container (Triton/vLLM) if not specified.
• API is secured via User Token (OAuth2/PAT).
• API schema follows standard formats (e.g., OpenAI-compatible for LLMs).

#Story 3.3: Interactive Model Widgets

• As a User,
• I want to test a model directly in the browser using an interactive widget,
• So that I can verify if it works for my data (e.g., OCR on a Greek document) before downloading it.

Acceptance Criteria:

• Widget input UI adapts to pipeline tag (Text, Image, Audio).
• Inference results are rendered visually (JSON pretty print, Image render).
• "Use in API" button generates a curl/python snippet for the current input.

#Story 3.4: Batch Inference with Datasets

• As a ML Engineer,
• I want to run batch inference using a trained model on a dataset with a custom script,
• So that I can process large volumes of data efficiently and implement domain-specific logic.

Acceptance Criteria:

• User can select a trained model and dataset from the platform when creating a batch inference job
• User provides or selects an inference script that defines how to apply the model to dataset entries
• System validates model compatibility with dataset format and provides clear error messages if incompatible
• Batch job is queued and executed on appropriate compute resources (K8s/Slurm) based on model requirements
• Progress tracking shows number of items processed, remaining, and any errors encountered
• Inference results are automatically saved to a specified output location (new dataset, model repository, or external storage)
• Logs and error reports are captured and accessible for debugging failed inference runs
• Resource usage (GPU/CPU hours, memory) is tracked and attributed to the project quota
• Batch inference job records link the source model, input dataset, and output results
• User can cancel running batch inference jobs with cleanup of partial results

#Epic 4: Dataset Management

Context: Efficient handling and previewing of large-scale datasets.

#Story 4.1: Dataset Streaming Preview

As a Data Scientist, I want to preview the first 100 rows of a large dataset in the browser without downloading it, So that I can verify data quality and schema before committing to a large transfer.

Acceptance Criteria:

• Viewer streams initial rows via API (no full download).
• Audio and Image columns render players/thumbnails directly in table cells.
• Pagination allows navigation through preview rows.

#Epic 5: Interactive Spaces (App Hosting)

Context: Hosting for demos and interactive ML applications (Gradio/Streamlit).

#Story 5.1: Space Creation & Auto-Build

• As a ML Researcher,
• I want to create a "Space" using Gradio, Streamlit, or Docker,
• So that I can demonstrate my model to stakeholder or any other user easily.

Acceptance Criteria:

• User selects SDK (Gradio/Streamlit/Docker) during creation.
• Pushing code triggers auto-build and deployment.
• requirements.txt dependencies are installed automatically.

#Story 5.2: CPU Allocation for Spaces

• As a User,
• I want to assign specific CPU hardware (or shared CPU) to my Space,
• So that I can run my demo application.

Acceptance Criteria:

• Setting a limit for CPU quotas for a consumer user that visit this space.
• Support for "sleep" configuration on idle paid Spaces.
• Shared GPU options for efficient resource usage.

#Story 5.3: Secrets Management

• As a Developer,
• I want to securely inject API keys (e.g., OpenAI tokens) into my Space,
• So that my code works but my secrets aren't exposed in the public repository.

Acceptance Criteria:

• UI for adding write-only secrets.
• Secrets injected as environment variables at runtime.
• Docker builds support secret mounting via /run/secrets/.

#Epic 6: Resource & Quota Management

Context: Managing the finite hardware resources and budget of projects.

#Story 6.1: Quota Requests

• As a Project Owner,
• I want to request initial quotas or extensions through a form,
• So that I can get the resources I need without out-of-band email communication.

Acceptance Criteria:

• Waldur interoperability
• Form captures requested GPU hours, Storage quotas, and justification.
• Request routes to Admin Dashboard for approval.
• System handles "Expired" freeze state for a grace period (read-only mode, keep only storage quotas, freeze CPU/GPU resources and jobs).
• Clean up after read-only mode after grace period, TODO: check with Panos Louridas
• After each project is approved, Themelio should call back to Pharos AAI to update the project claims for the specific extern_uid so we can propagate this information throughout the whole stack when the user logins again in the future.

#Story 6.2: Budget Monitoring & Alerts

• As a User,
• I want to see a dashboard of my resource usage and receive alerts,
• So that I don't unexpectedly run out of budget or leave expensive jobs running.

Acceptance Criteria:

• Waldur interoperability
• Dashboard shows Real-time Usage vs Limit (GPU hours, Storage).
• Auto-alerts sent at 50%, 80%, 95% usage thresholds.

#Epic 7: Compute & Training Orchestration

Context: Access to HPC and Cloud compute for heavy training and exploration.

#Story 7.1: Interactive Notebook Spawning

• As a Data Scientist,
• I want to spawn a JupyterLab instance on Kubeflow,
• So that I can interactively explore data and prototype models.

Acceptance Criteria:

• User selects hardware Kubeflow profile (GPU/CPU).
• User selects for predefined Kubeflow images.
• User defines their own custom images
• /home directory persists across stop/start cycles.
• Jupyterlab should include Git integration plugin attached.

#Story 7.2: Batch Job Submission

• As a ML Engineer,
• I want to submit batch training jobs to the Slurm/K8s cluster,
• So that I can train large foundation models that require long-running, massive compute.

Acceptance Criteria:

• Open OnDemand UI allows defining training Slurm jobs.
• Slurm Job is queued and scheduled based on priority/availability.
• User can cancel a running or queued job.

#Story 7.3: Job Observability

• As a User,
• I want to view real-time logs and metrics of my running jobs,
• So that I can debug failures or monitor training progress.

Acceptance Criteria:

• Live streaming of stdout/stderr Open OnDemand.
• Graphs for GPU utilization, Memory, and CPU usage.
• Option to attach a web-terminal to the running node/pod.

#Story 7.4: Model Training Workflow with Datasets

• As a ML Engineer,
• I want to train a new model using an existing dataset from the platform,
• So that I can leverage available data without re-uploading it and track my training experiments systematically.

Acceptance Criteria:

• User can select a dataset from the platform when configuring a training job
• Dataset is automatically mounted/accessible within the training environment
• Training job configuration captures dataset version, model architecture, and hyperparameters
• Training progress is tracked and can be monitored in real-time
• Training outputs (checkpoints, logs, metrics) are automatically saved to a model repository
• Trained model can be pushed directly to the Model Catalogue upon completion
• Training job records link to both the source dataset and the resulting model

#Epic 8: Experiment Tracking (MLOps)

Context: Tracking metrics, parameters, and artifacts for reproducibility.

#Story 8.1: MLflow Experiments

• As a Researcher,
• I want to log parameters and metrics to a project-specific MLflow instance,
• So that I can compare different runs and identify the best performing model.

Acceptance Criteria:

• MLflow server provisioned per project (or secure multi-tenant).
• UI allows visual comparison of runs.
• Runs are linked to specific Git commits and Dataset versions.

#Story 8.2: Artifact Promotion

• As a Engineer,
• I want to push a trained checkpoint from an experiment directly to the MLflow Model Registry,
• So that I can publish a successful training run without manual download/upload.

Acceptance Criteria:

• MLflow UI
• "Push to Registry" action available on Experiment/Run artifact.
• Artifacts are downloadable via UI.

Page source

Source snapshot · Revision f928834c8.