#ADR-0000: Friends & Family MVP Scope

Status: Proposed

Date: 2026-05-14

Deciders: TBD

Relates to: Product Requirements (Core), Functional Requirements, Architecture Overview

#Context

#What is the Friends & Family MVP

The Friends & Family MVP is a controlled early-access release of the Pharos AI Factory platform. Rather than a public launch, we are inviting a small, hand-picked group of users to work with the platform in a real-world setting — training models, running inference, and deploying interactive applications — while the team observes, supports, and iterates.

"Friends & Family" signals that these are trusted early adopters who understand the platform is not yet production-grade and are willing to provide candid feedback.

#Why now

The core ML workflows — project provisioning, data management, model training, inference, and Spaces — are functional end-to-end on the staging environment. However, they have only been exercised by the development team. A controlled release in June 2026 would validate these workflows with real users and real workloads before committing to a broader rollout.

The tension is between shipping quickly to gather feedback and shipping enough platform surface to make that feedback meaningful. Too narrow a scope and participants cannot complete realistic workflows; too broad and the team risks shipping unreliable features under time pressure.

#Who participates

The initial cohort consists of selected research teams, academic institutions, and SMEs with active ML workloads. Each participating organisation maps to one Themelio Project, and its members are onboarded through Pharos AAI with admin-assisted project setup.

#Success criteria

The Friends & Family MVP is successful if:

• End-to-end workflow completion — participants can register data, train or fine-tune a model, deploy it for inference, and build a Space on top, without hitting critical blockers
• Infrastructure stress test — the staging environment handles concurrent users and workloads, exposing bottlenecks and capacity limits before they become production incidents
• Operational lessons — the team gains hands-on experience operating the full stack (Waldur, GitLab, Kubernetes, Kubeflow, Slurm) with real users, surfacing gaps in monitoring, alerting, and incident response
• Actionable feedback — participants provide structured feedback that directly informs the roadmap for the next iteration

#Core Use Cases

The platform supports a complete ML lifecycle — from project setup and data preparation, through training and inference, to interactive demos. The subsections below follow the typical user journey.

#Project Provisioning & Resource Allocation

Before any ML work can begin, a Themelio Project must be created and approved. Each project provisions an isolated workspace across all platform services.

Resource requests cover both Cloud (Kubernetes) — explicit CPU, RAM, and storage quotas — and HPC — node-hour allocations. Pharos admins review and approve requests through the Waldur admin interface, and a Waldur site agent handles automated provisioning of GitLab groups, K8s resources, and Kubeflow profiles.

For detailed requirements, see Themelio Projects.

#Catalogue & Resource Discovery

The Themelio Catalogue is the central registry for models, datasets, spaces, and recipes. Users can browse and search existing resources before creating their own.

• Search & filter by task type, library, language, license, and tags
• Model cards and dataset cards display structured metadata (description, version, license, metrics) auto-parsed from README.md YAML front matter
• Visibility controls — resources can be Public, Gated, or Private; private resources are hidden from public search
• Engagement metrics — downloads, likes, and last-updated timestamps help surface popular resources

For detailed requirements, see Models and Data Management.

#Data Management

Datasets are Git-backed repositories with full version history and support for large binary files via LFS.

• Upload data via standard git push (HTTPS or SSH); large files (up to TB-scale) handled transparently through LFS with chunk-level deduplication
• Version datasets using Git commits — every change is tracked with immutable history
• Structured metadata — format (CSV, JSON, Parquet, Arrow, etc.), size, modality, languages, and license are captured at registration

If core features are delivered ahead of schedule, the following are stretch goals for the Friends & Family MVP:

• Preview — stream the first 100 rows of a dataset, rendering text inline in the browser
• Quota enforcement — enforce storage quota limits at upload time

For detailed requirements, see Data Management.

#Training & Fine-tuning

Users can train or fine-tune models using two complementary approaches, depending on the scale and nature of the work:

In all cases, datasets registered on the platform are automatically accessible within the training environment. Training outputs (checkpoints, metrics, logs) can be pushed to the Themelio Catalogue as model repositories. Experiment tracking via MLflow is not part of the Friends & Family MVP and is deferred to a future release.

For detailed requirements, see Compute & Training Orchestration and Experiment Tracking.

#Inference

Once a model is registered in the catalogue, users can run batch inference by selecting a model, a dataset, and an inference script — the platform runs the job on K8s or Slurm and stores results back to the platform.

API-based inference (one-click REST endpoints) and interactive browser widgets are planned for a future release but are not part of the Friends & Family MVP.

For detailed requirements, see Models and Interactive Spaces.

#Spaces — Interactive Applications

Spaces are deployed interactive applications that serve as demos, frontends, or tools built on top of platform resources. Users write code in a Git repository, and the platform builds and runs it as a containerised web app with a public URL.

For the Friends & Family MVP, only Gradio Spaces are supported: push code with a requirements.txt and the platform auto-builds and deploys the application. Support for Streamlit and Docker-based Spaces is planned for a future release.

Secrets (API keys, tokens) are managed through a write-only UI and injected as environment variables at runtime.

For detailed requirements, see Interactive Spaces.

#Recipes & Upskilling Materials

Each project is provisioned with dedicated GitLab subgroups for recipes and upskilling materials. For the Friends & Family MVP, these function as plain Git repositories — teams can store notebooks, guides, and reference scripts. A structured catalogue experience for recipes (search, metadata cards, discovery) is planned for a future release.

#Considered Options

#Option A: Conservative MVP

Ship the minimum set of features that enables end-to-end ML workflows: project provisioning, catalogue browsing, model and dataset registration, interactive notebooks (Kubeflow), HPC job submission (Slurm), batch inference, Gradio Spaces, and auth hardening. Onboarding is manual and admin-driven. Features that require unresolved integration work (experiment tracking, proposal workflow, SSH access, storage billing, Open OnDemand, Cloud batch jobs, API-based inference, interactive widgets, Streamlit/Docker Spaces) are deferred.

Pros:

• Focuses engineering effort on reliability and polish of core workflows
• Avoids shipping half-integrated features (e.g., MLflow) that could confuse early adopters
• Manual onboarding is acceptable for a cohort of tens of users
• Leaves room for bug fixes and operational hardening before June

Cons:

• Participants must track experiments manually (local logs, spreadsheets)
• No self-service resource requests — all quota changes go through a Waldur admin
• HPC onboarding remains partially manual (Slurm account creation requires a GRNET admin)
• Limited platform surface may not stress-test all integration points

#Option B: Extended MVP

Ship everything in Option A plus experiment tracking (MLflow), self-service proposal workflow, direct SSH access to HPC, storage billing and quotas visibility, and Open OnDemand integration.

Pros:

• More complete platform experience for participants
• Tests more integration points with real users
• Reduces manual admin overhead (self-service proposals, automated HPC onboarding)

Cons:

• MLflow integration approach is still under exploration — shipping it risks a half-baked experience
• Proposal workflow requires Waldur API integration that is not yet validated in staging
• SSH access and Open OnDemand require coordination with the HPC operations team, adding external dependencies to the June timeline
• Broader scope increases the risk of shipping unreliable features and dilutes engineering focus

#Decision

We will ship Option A: Conservative MVP for the Friends & Family launch in June 2026. The core platform capabilities and the onboarding process are described below. For full functional specifications, see Functional Requirements.

#Project Resource Management

Pharos Admins will create and manage projects through Waldur, with resources provisioned automatically via Waldur site agent when orders are approved. Each project gets an isolated workspace across GitLab (model, dataset, space, and code repositories), Kubernetes (compute namespaces with ResourceQuotas, Roles, RoleBindings, ServiceAccounts, PVCs), and HPC (Slurm accounts). See Themelio Projects.

#Workshop Onboarding

For the initial Friends & Family workshop, project setup follows a manual, admin-driven process:

1. Gather participants. Pharos SA compiles a list of organisations (e.g. NTUA) and their members (email addresses — academic login or Google OIDC). Each organisation maps to one Themelio Project.
2. Invite users via AAI. For members not yet registered in Pharos AAI:
   • AAI sends an email invitation prompting account activation
   • User clicks the invitation link and authenticates with Pharos AAI
   • User agrees to Terms & Conditions and project participation
   • Pharos Admin is notified to add the newly-activated user to the project
3. Create projects manually. A Pharos Admin creates each project through the Waldur UI, adds members with correct RBAC, and creates orders from:
   • Cloud offering (K8s): triggers the Waldur site agent to provision GitLab groups, K8s resources (namespace, ResourceQuotas, Roles, RoleBindings, ServiceAccounts, PVCs)
   • HPC offering (Slurm): executed manually by an authorised GRNET HPC admin
4. Users log in. Members visit the staging instance, authenticate with Pharos AAI, and see the project corresponding to their organisation. All members can create Themelio resources (models, datasets, spaces, repos) under the project.

#Model & Dataset Registration

Users register and manage models and datasets with structured metadata, edit entries through dedicated forms, and browse detail pages within their project context. See Models and Data Management.

#Multi-tenant Notebooks

Kubeflow Profiles synced with Waldur Projects via the Waldur site agent (Kubeflow component), providing isolated notebook environments with GPU access using Cloud quotas. See Compute & Training Orchestration.

#Cloud, HPC & Slurm Integration

Automated Slurm job submission through repository-defined workflows (actions.yaml), per-user SSH authentication for HPC access, and job status tracking with output streaming. See Compute & Training Orchestration.

#Job & Deployment Reliability

Cancel running K8s deployments and Slurm jobs, persistent job history (replacing ephemeral in-memory tracking), and storage volume purging to avoid "no space left on device" on long-running deployments. Job queueing with automatic retry on transient failures is a stretch goal.

#Authentication & Security

Improved token validation and session management, tighter permission scopes for service credentials, and clear session expiration handling.

#Open Questions

#Q1: Pre-warmup provisioning

GitLab and K8s accounts are created on first login. Can we pre-provision accounts (GitLab invite, Waldur invite, K8s ServiceAccount, Slurm account) before the workshop to avoid first-login delays?

#Q2: Stop and destroy deployments

Investigate stop and destroy features for K8s deployments based on scaling down replicas and destroying pods.

#Q3: Storage capacity in deployments

Add support for defining storage capacity in deployment definitions (actions.yaml), and inform users about MVP storage limitations.

#Q4: Demo resources

Create demo Themelio resources (models, datasets, spaces, repos) for workshop participants to explore.

#Consequences

#Positive outcomes

• Real-world validation. The team gets the first external signal on whether the core ML lifecycle works end-to-end outside of internal testing.
• Focused engineering effort. By deferring half-integrated features, the team can invest in reliability, monitoring, and operational readiness for the features that ship.
• Feedback-driven roadmap. Participant feedback will directly determine which deferred capabilities to prioritise next.

#Accepted trade-offs

• Manual onboarding doesn't scale. The admin-driven workshop setup (AAI invitations, manual Waldur project creation, manual HPC provisioning) works for a cohort of tens of users but becomes a bottleneck beyond that. This is acceptable for Friends & Family — the proposal workflow will replace it for general availability.
• No experiment tracking means external workarounds. Without MLflow integration, participants must track training runs manually (local logs, spreadsheets, or their own MLflow instances). This limits reproducibility during the MVP but avoids shipping a half-integrated tracking solution.
• No self-service resource requests. Users cannot request or adjust quotas through Themelio — all changes go through a Waldur admin. This adds latency to resource adjustments but keeps the MVP scope manageable.
• HPC onboarding remains partially manual. Slurm account creation requires a GRNET HPC admin. This is a known operational bottleneck that will be addressed when the Slurm site agent is automated.

#Deferred to post-MVP

These capabilities will be addressed as capacity allows after June 2026:

• Proposal workflow — connect resource requests to Waldur's proposal management system, replacing manual admin provisioning
• Direct SSH access to HPC — SSH key management through the platform, propagated via AAI; requires coordination with the HPC operations team
• Platform resilience & capacity planning — systematic stress testing of the staging environment to understand limits and validate rollback procedures
• Storage billing & quotas — visibility into storage consumption across Git repositories, LFS objects, and build artifacts, reported back to Waldur
• Experiment tracking (MLflow) — integration approach still under exploration; if MLflow Workspaces integrate cleanly with our project model, teams would get experiment tracking out of the box
• Open OnDemand — browser-based access to HPC resources for interactive sessions; depends on alignment with the HPC operations team
• Cloud batch jobs (Kubernetes) — define training jobs via actions.yaml submitted and scheduled on Kubernetes; deferred in favour of HPC-only batch training for the MVP
• API-based inference — one-click REST API deployment with serving containers (e.g. vLLM, HuggingFace TGI) and OpenAI-compatible endpoints
• Interactive inference widget — auto-generated browser widget for quick model evaluation by task type
• Streamlit & Docker Spaces — additional Space runtimes beyond Gradio

#What we expect to learn

• Capacity baselines — real usage patterns will inform resource defaults, quota sizing, and autoscaling thresholds for production.
• Onboarding friction points — first-login delays (GitLab/K8s account creation), AAI invitation flows, and project discovery UX will be stress-tested with real users for the first time.
• Operational gaps — monitoring, alerting, and incident response procedures will be validated under realistic conditions, exposing blind spots before a broader rollout.
• Product priorities — participant feedback will directly inform which deferred capabilities to prioritise next.

#References

• Functional Requirements — detailed specifications for all platform capabilities
• Architecture Overview — system structure and service interactions
• Waldur — project and resource management platform
• Kubeflow — ML toolkit for Kubernetes
• Slurm — HPC workload manager

Page source

Source snapshot · Revision f928834c8.