Themelio compatibility reference. This page describes the retained Themelio implementation or its historical design. Check the parallel-operation guide before applying it to the current platform.

#DD-0021: Themelio CLI — Implementation

Related ADR: ADR-0021: Themelio Command-Line Interface

Status: Draft

Date: 2026-07-06

Authors: TBD

#ADR-derived technical notes

Implementation detail that surfaced while authoring ADR-0021. A complete working specification also exists at plans/themelio-cli/SPEC.md and is the primary input for this DD.

• Package / stack: Python 3.11+, console entry point themelio, distributed via a curl … | sh standalone installer and pipx install themelio-cli / uvx themelio. Stack: Typer (commands), httpx (HTTP + SSE), rich (tables/progress), PyYAML (pre-parse of actions.yaml), keyring (token storage). git/git-lfs are the user's, not a runtime dependency.
• Auth / config precedence: --host/--token flags → THEMELIO_HOST/ THEMELIO_TOKEN → ~/.themelio/config (or keyring) → built-in default host https://dev.themelio.pharos.grnet.gr. Single PRIVATE-TOKEN header.
• git credential helper: git-setup prints git config --global credential.https://<host>.helper '!themelio git-credential' by default; --apply writes it. Hidden themelio git-credential speaks git's credential-helper protocol (emits username=oauth2 + password=<token>).
• Content commands print git, never shell out; no single-file API shortcut.
• New endpoint POST /api/actions/validate reuses the web action library and delegates ref: resolution to k8s-root-actor (populateRefs).

#Overview

This document specifies how to build the themelio command-line client: a standalone Python package that is a thin, typed client of the Themelio web API, with a themelio <type> <ns/repo> <verb> grammar, keyring-backed api-scoped PAT auth, git hand-off for content transfer (via a git-setup credential helper), and action-driven compute that yields observe-only jobs and deployments. It also specifies the one new backend endpoint the CLI depends on, POST /api/actions/validate.

#Goals and Non-Goals

#Goals

• Installable two ways: a zero-prerequisite curl … | sh standalone installer, and pipx install themelio-cli / uvx themelio for Python users — both yielding a themelio command covering auth, repository lifecycle, actions, and job/deployment observation against the existing web API.
• Resource-first command grammar with the shared verb set, implemented cleanly on top of Typer/Click.
• One credential (api PAT) for both API and git; stored in the OS keyring.
• Scriptable output (--json, --quiet) and stable exit codes.
• The POST /api/actions/validate endpoint, reusing existing web/k8s-root-actor logic.

#Non-Goals

• Moving repository bytes through the CLI (delegated to the user's git/git-lfs).
• A browser device-login flow (deferred; see Open Questions).
• Repo deletion, file-level commits, branches/tags, and a local content cache (unsupported by the backend today).

#Design

#1. Dependencies

Build backend: hatchling, src/ layout, requires-python = ">=3.11".

#2. File structure

themelio-cli/
├── pyproject.toml                 # metadata, deps, [project.scripts] themelio = "themelio.cli:app"
├── README.md
├── src/themelio/
│   ├── __init__.py                # __version__
│   ├── __main__.py                # enables `python -m themelio`
│   ├── cli.py                     # root Typer app; global options; assembles sub-apps
│   ├── config.py                  # Settings model + host/token resolution & persistence
│   ├── keyring_store.py           # keyring get/set/delete with file fallback
│   ├── client.py                  # ThemelioClient: httpx wrapper, PRIVATE-TOKEN, error mapping
│   ├── errors.py                  # exception hierarchy ↔ exit codes (plain exceptions)
│   ├── context.py                 # AppContext (settings/fmt/repo) + repo resolution helpers
│   ├── output.py                  # render()/detail(): rich tables vs --json/--quiet
│   ├── normalize.py               # strip GitLab plumbing; expose namespace/repo refs
│   ├── models.py                  # pydantic response models (Repo, Job, Deployment, Action…)
│   ├── restypes.py                # resource-type constants + plural/singular mapping
│   ├── grammar.py                 # ResourceGroup: the <type> <ns/repo> <verb> parser
│   ├── gitcmd.py                  # repo git URL, clone/upload command builders
│   ├── sse.py                     # SSE line iterator over httpx streaming (future --follow)
│   └── commands/
│       ├── __init__.py
│       ├── auth.py                # login · logout · whoami · token · list-tokens · revoke
│       ├── git_setup.py           # git-setup (+ hidden git-credential)
│       ├── resource.py            # make_resource_app(): shared verbs per {type}
│       ├── actions.py             # make_actions_app(): ls · describe · run · validate
│       ├── compute.py             # make_compute_app(): shared jobs/deployments verbs
│       ├── jobs.py                # thin wrapper: make_compute_app("jobs", …)
│       └── deployments.py         # thin wrapper: make_compute_app("deployments", …)
└── tests/
    ├── conftest.py                # temp config, disabled-keyring, invoke helper (respx)
    ├── test_grammar.py            # repo-first vs repo-less parsing
    ├── test_config.py             # precedence + keyring fallback + 0600
    ├── test_auth.py · test_resource.py · test_actions.py
    ├── test_jobs.py · test_deployments.py
    └── test_git_setup.py          # printed config + credential-helper protocol I/O

The error hierarchy is plain exceptions (not click.ClickException) because the pinned Typer vendors its own Click; a root RootGroup(TyperGroup) catches ThemelioError in invoke() and raises typer.Exit(code), giving stable exit codes under both real runs and CliRunner.

#3. Command grammar (grammar.py)

The resource-first grammar themelio <type> <ns/repo> <verb> is not Click's default (group verb args). A repo reference always contains exactly one /; verbs never do. We exploit that with a custom group that pops a leading repo token into the context before resolving the verb:

import re
from typer.core import TyperGroup

REPO_RE = re.compile(r"^[^/\s]+/[^/\s]+$")

class ResourceGroup(TyperGroup):
    def parse_args(self, ctx, args):
        # `models vkefallinos/bert info` → repo popped, subcommand = info
        # `models ls` / `models actions validate` → nothing popped
        if args and REPO_RE.match(args[0]) and args[0] not in self.commands:
            ctx.ensure_object(dict)["repo"] = args.pop(0)
        return super().parse_args(ctx, args)

• Each resource sub-app is typer.Typer(cls=ResourceGroup); commands read the repo via ctx.obj["repo"] (Typer injects ctx: typer.Context).
• ls (no repo) and the actions/jobs/deployments groups dispatch normally because their first token has no /.
• Those sub-apps are mounted on the resource app, so models vkefallinos/bert actions run train (or … jobs ls) pops the repo at the resource level and reaches the nested verb with ctx.obj["repo"] set; models actions validate f never pops a repo, and repo-scoped verbs raise a usage error (exit 2) when repo is absent.

#4. Root app & resource factory

cli.py builds the root Typer app, defines global options as a top-level callback, and mounts sub-apps:

app = typer.Typer(no_args_is_help=True)

@app.callback()
def main(ctx, host: str = typer.Option(None, envvar="THEMELIO_HOST"),
         token: str = typer.Option(None, envvar="THEMELIO_TOKEN"),
         json_: bool = typer.Option(False, "--json"),
         quiet: bool = typer.Option(False, "-q", "--quiet"),
         yes: bool = typer.Option(False, "-y", "--yes")):
    ctx.obj = AppContext(settings=resolve_settings(host, token), fmt=..., assume_yes=yes)

app.add_typer(auth.app, name="auth")
app.command("git-setup")(git_setup.git_setup)
for t in ("models", "datasets", "spaces", "repos"):
    app.add_typer(make_resource_app(t), name=t)
app.add_typer(jobs.app, name="jobs")                # top-level (--repo / --namespace)
app.add_typer(deployments.app, name="deployments")

models/spaces/repos additionally mount the same jobs.app/deployments.app as repo-scoped shortcuts (e.g. themelio models vkefallinos/bert jobs ls), so the verbs run with the repo already in context.

make_resource_app(resource_type) (in commands/resource.py) returns a ResourceGroup Typer with the shared verbs, closing over resource_type so all four types share one implementation. has_compute = type in {models, spaces, repos} mounts the actions, jobs, and deployments sub-apps (datasets get none); has_search = type in {models, datasets} toggles --search. The jobs/ deployments sub-apps are the same Typer apps as the top-level jobs.app/ deployments.app — mounting them under a ResourceGroup means the repo is popped into ctx.obj["repo"] before their verbs run, so those verbs read the repo from context instead of requiring --repo. Verb → endpoint mapping:

#5. HTTP client (client.py)

A thin wrapper around one reused httpx.Client:

class ThemelioClient:
    def __init__(self, settings):
        self._c = httpx.Client(base_url=settings.host,
                               headers={"PRIVATE-TOKEN": settings.token},
                               timeout=httpx.Timeout(30.0, read=None))
    def request(self, method, path, **kw) -> Any:
        r = self._c.request(method, path, **kw)
        if r.is_success: return r.json() if r.content else None
        raise error_from_response(r)   # maps status → typed exception
    def stream_sse(self, path): ...    # see §9

error_from_response maps 401/403 → AuthError, 404 → NotFoundError, 409 → ConflictError, 400 → ValidationError (carrying fieldErrors), else ThemelioError. Requests without a resolved token short-circuit to AuthError ("run themelio auth login").

#6. Config & credential storage (config.py, keyring_store.py)

• Settings is a pydantic model (host: HttpUrl, token: SecretStr | None).
• resolve_settings() applies the ADR precedence: flags → env → config file → default host https://dev.themelio.pharos.grnet.gr (token has no default).
• Config file ~/.themelio/config (TOML, mode 0600) stores host only when the token lives in the keyring; if no keyring backend exists it also stores token.
• keyring_store wraps keyring.get/set/delete_password("themelio", host), catching keyring.errors.NoKeyringError to fall back to the config file. logout clears both.

#7. Content transfer (gitcmd.py, commands/resource.py, git_setup.py)

• repo_git_url(host, type, ns, repo) -> f"{host}/{type}s/{ns}/{repo}.git".
• download/upload print the exact commands (never execute git): clone → git clone <url>; upload → cd <repo> && git add -A && git commit -m … && git push, prefixed with a one-time themelio git-setup --apply && git lfs install hint.
• git-setup (commands/git_setup.py):
  • Default: print git config --global credential.https://<host>.helper '!themelio git-credential'.
  • --apply: run it via subprocess.run([... "git", "config", ...]).
  • Hidden themelio git-credential command implements the credential-helper protocol: read key=value lines from stdin until blank; if host matches the configured host, print protocol=https, host=<host>, username=oauth2, password=<token-from-keyring>. store/erase verbs are no-ops (the token is owned by themelio auth, not git).

#8. Actions (commands/actions.py)

• actions ls / describe read the actions array embedded in GET /api/{type}/{ns}/{repo} (there is no list endpoint); describe renders the whole action as a panel — its type, the job/deployment it yields, and every parameter (type, required, default, enum options, description). Note the info payload exposes only name/title/label/type/parameters per action; action-level options (cpus/gpus/ image/script) are not returned, so describe cannot show them.
• actions run <action> [--param k=v]… [--branch main] [--follow] → POST /api/{type}/{ns}/{repo}/action/{branch} with {action, parameters} (route exists for models/spaces/repos only). The web route relays the k8s-root-actor response verbatim (201 create / 409 already-running); the id is embedded in that upstream payload, so the CLI extracts it defensively (tries id/jobId/deploymentId/name and a nested job/deployment). Whether it is a job or deployment is decided by the action type (slurm-job → job, else deployment); on --follow the CLI points at the relevant logs command.
• actions validate [file=./actions.yaml] → offline yaml.safe_load sanity check, then POST /api/actions/validate with the raw content; prints errors/warnings and exits non-zero when valid is false.

actions is read/run-only: there is no init verb. Authoring actions.yaml (and any commit to it) is done in the repository itself, not through the CLI.

#9. Jobs, deployments & log streaming (commands/jobs.py, deployments.py, sse.py)

• Observe-only verbs: ls [--repo|--namespace] [--status], get <id> --repo, logs <id> --repo [--follow], wait <id…> --repo [--timeout], destroy <id> --repo. These map to the repo-scoped …/{jobs,deployments}[/{id}] routes and the namespace routes /api/{jobs,deployments}/namespaces/{ns}.
• Repo resolution: each verb resolves the repo from ctx.obj["repo"] first (set when reached via the <type> <ns/repo> jobs|deployments … shortcut), else the --repo option. The same jobs.app/deployments.app therefore serve both the top-level and the per-resource shortcut with no duplicated code; --namespace listing is only valid on the top-level form.
• wait is client-side polling of get until a terminal status, honoring --timeout; exit non-zero if any target failed.
• logs: there is no separate log-stream endpoint; the get response (…/{jobs,deployments}/{id}) embeds an ordered logs array of {id, timestamp, level, message, source_status}. logs reads that array and prints each line timestamped, coloring error/warn and dimming empty-message status transitions. This is identical for jobs and deployments — both k8s-root-actor GET /jobs/:id and GET /deployments/:id return the ordered logs. --follow re-fetches every 2 s and prints only lines not seen (tracked by id), stopping at a terminal status. --json emits the raw logs array. (The spaces/{ns}/{repo}/logs/run route — a hardcoded stub — is not used. sse.py remains for a future true-stream endpoint, when --follow can switch from polling to SSE.)

#10. Output & exit codes (output.py, errors.py)

• render(data, *, columns) picks human tables (rich) by default, raw JSON with --json, and id-only lines with --quiet.
• Platform-native view (normalize.py). The web API leaks its GitLab backing store — list items carry a GraphQL global id (gid://gitlab/Project/74) rather than a usable reference, plus gitlabUrl/namespaceName/cardData/slug and, on jobs/ deployments, gitlab_project_id/gitlab_user_id. The CLI strips these in both human and --json output and surfaces the canonical namespace/repo reference (from the slug) as id, so ls leads with a copy-pasteable Repository column and -q prints refs usable in the next command. Topics/tags are preserved verbatim. Jobs/deployments tables read the real API fields (action_name → Action, created_at → Created).
• Collection vs repo verbs. ls is a collection verb (whole type); giving it a repo (<type> <ns/repo> ls) is rejected with exit 2 pointing at ls-files. On the jobs/deployments repo-scoped mount, --repo/--type are hidden (the repo comes from the path and the type is fixed); they remain on the top-level mount.
• The root RootGroup.invoke() catches the exception hierarchy and converts it to a rendered message + typer.Exit(code): 0 ok · 2 usage · 3 auth · 4 not-found · 5 conflict · 6 validation (renders fieldErrors per parameter) · 1 other.

#Data Model

Config file (~/.themelio/config, TOML):

host  = "https://dev.themelio.pharos.grnet.gr"
token = "…"   # present only when no OS keyring backend is available

Keyring entry: service themelio, username <host>, password <api-scoped PAT>.

Pydantic response models in models.py mirror the API payloads the CLI renders (Repo, FileEntry, TokenInfo, WhoAmI, Job, Deployment, Action, ActionParameter, ValidationResult). Unknown fields are ignored (model_config = ConfigDict(extra="ignore")) so the CLI tolerates API additions.

#API Design

#New: POST /api/actions/validate (web service)

New route services/web/src/app/api/actions/validate/route.ts. Auth: standard PRIVATE-TOKEN (no repo scope — validates content).

Request:

{ "content": "<actions.yaml text>", "action": "train-model" }

Response:

{ "valid": false,
  "errors":   [{ "action": "train-model", "path": "parameters.epochs.type",
                 "message": "unsupported type 'number' (use string|enum)" }],
  "warnings": [{ "action": "train-model", "path": "script",
                 "message": "$foo references an undefined parameter" }] }

Implementation (src/lib/actions/action-validator.ts holds the reusable logic; the route orchestrates):

1. Require private-token (401 if absent). Parse via a new parseActionsForValidation() that calls js-yaml directly and surfaces errors — unlike the executor's parseActionsYaml/loadActionsYaml, which swallow them and return {}. Malformed YAML or a non-mapping top level → 400.
2. validateActionsStructure() reuses action-parameters.ts (SUPPORTED_PARAMETER_TYPES, validateParameterValue) and action-types.ts for structural + per-parameter checks: parameter type ∈ string|enum, enum options/enum present and default a member, action type ∈ slurm-job|k8s-deployment|k8s-inferenceservice, and $name (incl. hyphens) reference warnings (undefined refs / unused params). Invalid content returns 200 with valid:false + issues (only malformed input is a 400).
3. For each action carrying a ref:, kubeFetch('/actions/validate', …) (base K8S_ROOT_ACTOR_URL from src/config.ts) posts { actionConfig } to the new k8s-root-actor route POST /actions/validate (src/routes/actions-validate.ts, registered in src/index.ts). That route runs populateRefs (src/utils/actions-yaml-references.ts), maps its error classes to structured { path, message } issues, and checks the resolved type plus a config-free GPU coherence rule (gpuLabel required when gpus set). The gpuLabel allow-list check is intentionally left to the executor at run time so the validate route stays decoupled from DB-bound config. Its issues merge into the web response.

The web authoring UI ("Setup deployment") can call the same endpoint, keeping one source of truth.

#Existing endpoints consumed

whoami-v2 (unwrapped UserProfile); settings/tokens[/{id}] (POST body {name, scopes[], expiresAt?} → {message, token}; GET → bare array); {type} (list; author is ?namespace=); {type}/search?search= (models/datasets only — no spaces/search); {type}/{ns}/{repo}[/readme|/tree/{rev}|/settings]; repos/create (singular type); fork/{…path} (catch-all, body {namespacePath}); {type}/{ns}/{repo}/action/{branch} (models/spaces/repos; response relayed from k8s-root-actor); and the {jobs,deployments} repo- and namespace-scoped routes, whose …/{id} detail embeds the ordered logs array the logs verb reads. The spaces/{ns}/{repo}/logs/run stub is not consumed. No changes required to any of these; the CLI is coded against these verified shapes.

#Testing Strategy

• Grammar (test_grammar.py): ResourceGroup.parse_args pops repos only when a / token precedes a verb; ls and actions validate never pop; missing-repo on a repo-scoped verb exits 2.
• Commands (test_resource/actions/jobs/deployments.py): drive Typer's CliRunner with respx-mocked httpx routes; assert method+path+body and rendered output for human/--json/--quiet, and exit-code mapping for 401/404/409/400.
• Config (test_config.py): precedence order; keyring fallback via a monkeypatched NoKeyringError.
• git-setup (test_git_setup.py): printed config string; git-credential reads a stdin request and emits username=oauth2 + password for the matching host only.
• Endpoint: web-side action-validator.test.ts (node:test) covering malformed/non-mapping YAML, unsupported action/parameter types, enum options/default, $name warnings, and the single-action filter; plus k8s-root-actor routes/actions-validate.test.ts (Fastify inject) exercising populateRefs success, a missing ref, an unsupported type, and a missing body.
• CI: CLI — ruff, mypy, pytest (3.11/3.12) + wheel build; backend — pnpm test in services/web and services/k8s-root-actor.

#Deployment & Rollout

• Package: themelio-cli in its own repository (Python is outside the pnpm monorepo build). Two distribution channels, both producing the same themelio command:
  • Python users: wheel/sdist on the internal index / GitLab Package Registry — pipx install themelio-cli or uvx themelio (isolated environments).
  • Zero-prerequisite: curl -LsSf https://<host>/cli/install.sh | sh (and install.ps1 for Windows). The script detects OS/arch, downloads the matching PyInstaller single-file executable from the release artifacts, installs it to ~/.local/bin/themelio (%LOCALAPPDATA% on Windows), chmod +x, and prints PATH guidance. themelio --version self-reports the channel.
  • CI build matrix (GitLab CI) produces pyinstaller --onefile binaries for linux x86_64/aarch64, macOS arm64/x86_64, and Windows x86_64, plus the wheel; the install.sh/install.ps1 scripts and binaries are attached to each tagged release. The credential helper works under both channels because git invokes themelio git-credential from PATH regardless of how it was installed.
• Backend: the POST /api/actions/validate route ships with the normal web deploy; the companion k8s-root-actor POST /actions/validate route ships with its service. Both are additive and backward-compatible.
• Prerequisite already shipped: the Settings token UI exposes the api scope.
• Phasing (from the spec): M1 read + auth + git-setup; M2 write + actions (needs the validate endpoint); M3 log streaming, keyring, completions, polish.
• Rollback: removing the CLI has no server impact; the two new endpoints can be withdrawn independently since nothing else depends on them.

#Open Questions

1. Distribution channel? Resolved: ship both — PyInstaller single-file binaries behind a curl … | sh installer (à la hf) and a wheel via pipx/uvx. Open sub-question: which host serves /cli/install.sh and the release artifacts (the web host vs the GitLab release page).
2. Should POST /api/actions/validate also accept a repo + ref to validate an already-committed actions.yaml, in addition to raw content?
3. Device-login flow (reusing OIDC + impersonation tokens) as a future auth login mode — out of scope for v1, tracked in ADR-0021.

#References

• ADR-0021: Themelio Command-Line Interface
• ADR-0012: Job and Deployment Management System
• ADR-0013: actions.yaml Automation System
• Full command specification: plans/themelio-cli/SPEC.md
• Typer · httpx · keyring · Rich
• git credential helper protocol