Themelio compatibility reference. This page describes the retained Themelio implementation or its historical design. Check the parallel-operation guide before applying it to the current platform.

#DD-0006: Automated Seeding Strategy — Implementation

Related ADR: ADR-0006: Automated Data Seeding Strategy

This document provides the full technical specification for implementing the automated seeding strategy described in ADR-0006. It is intended to be followed by an engineer or an automated agent.

#1. Kubernetes Seeder Job

A single-run Kubernetes Job deployed by Ansible. The Job is limited to Git repository mirroring; it does not run Waldur orchestration, does not poll site-agent logs, and does not need Kubernetes API access.

#Container design

• Base image: alpine:latest (no pre-installed tools required)
• Main container: installs required tools at startup with apk add --no-cache bash git git-lfs curl jq, then runs the embedded repository mirroring script.
• Ephemeral storage: an emptyDir volume at /workspace (default sizeLimit: 10Gi) is used for cloning repositories. Each repository directory is deleted after pushing to keep peak usage bounded.
• Script delivery: Ansible renders seeder-job.yml.j2, which produces a complete Kubernetes Job manifest with the bash script embedded directly in .spec.template.spec.containers[0].command and runtime values passed through .spec.template.spec.containers[0].env. There is no ConfigMap or separate seeder credentials Secret — the final Kubernetes resource contains the script and container environment.
• Kubernetes access: The Job sets automountServiceAccountToken: false and does not define a custom ServiceAccount, Role, or RoleBinding.

#Workflow

The workflow is split between the Ansible role and the Kubernetes Job.

#Ansible-controlled Waldur orchestration

1. Reject a stale seeder Job. Before starting, check whether Job/themelio-seeder already exists in the themelio namespace. If it exists, fail clearly and leave the existing Job untouched.

2. Validate the external GitLab token. Read SEEDER_EXTERNAL_GITLAB_TOKEN from the local environment of the operator running Ansible. Validate it with GET https://gitlab.grnet.gr/api/v4/user. If it is missing, invalid, or expired, fail before creating any Kubernetes Job.

3. Run Waldur operations through kubectl exec. The role uses kubectl -n waldur exec -i deploy/waldur-mastermind-worker -- waldur shell so Waldur-side operations run inside the application context.

4. Validate the Waldur organization exists. Look up an organization named pharos. If it exists, retrieve and store its UUID for use in subsequent steps. If it does not exist, fail with: organization does not exist, run waldur-offering target first.

5. Ensure pharos-admin exists in Waldur. Create or update the user if needed. This user is a Waldur staff user, has a password from vault, and is granted access to the seeded project.

6. Ensure the Waldur project exists. Create or reuse the project and grant pharos-admin project-level access. The project name and project slug are both pharos.

7. Place or reuse the order. Look up the Cloud offering by the shared waldur_offering_name inventory variable and use the resulting UUID to initiate or reuse an order. The role only reuses an existing order/resource for the configured allocation name if the order is not a terminate order and the linked resource is not terminated, terminating, erred, or deleted. Terminated resources and terminate orders are ignored so a fresh create order can be placed. The order uses these values:

   Field	Description
   ORGANIZATION	Waldur organization (pharos)
   PROJECT	Waldur project name and slug (pharos)
   RESOURCES	Resource allocations: 10 CPU, 12 GB memory, 20 GB storage
   ALLOCATION NAME	Human-readable name for the order
   DESCRIPTION	Free-text description of the allocation
   TERMINATION DATE	Configurable end date for the resource allocation; empty means no expiry

   The role creates the order through Waldur's in-process DRF API client, using compact UUID hyperlinks. It obtains the plan URL from the serialized marketplace-public-offerings response instead of constructing the plan URL by hand, sets accepting_terms_of_service: true, and attempts approve_by_provider and approve_by_consumer after order creation. The returned order ID is cached as an Ansible fact for the current run. Subsequent runs inspect existing Waldur resources and orders instead of relying on stored local state. This order triggers the waldur-k8s-site-agent to begin provisioning GitLab subgroups, users, and the Kubernetes namespace with resource quotas.

8. Wait for waldur-k8s-site-agent provisioning. After placing a new order, provisioning is handled asynchronously by waldur-k8s-site-agent. The agent creates GitLab subgroups and users. The role resolves the current pod for the waldur-agent-cloud-order-process deployment and scans that pod's logs for the returned order ID until processing completes. It searches for both compact and hyphenated UUID forms and treats the order as complete only when the matching logs contain The order <hyphenated-order-uuid> has been successfully processed. If the role reuses an existing complete order with a non-terminated resource, this polling step is skipped.

   The agent emits structured JSON log events. Example events:

   {
     "event": "Processing order themelio-seed-allocation (d08dd31f-8ae7-483b-8d76-c40df6cfb9a3) type Create, state executing",
     "level": "info",
     "logger": "waldur_site_agent.backend",
     "timestamp": "2026-05-29T16:32:37.250450+00:00"
   }
   {
     "event": "The order d08dd31f-8ae7-483b-8d76-c40df6cfb9a3 has been successfully processed",
     "level": "info",
     "logger": "waldur_site_agent.backend",
     "timestamp": "2026-05-29T16:32:42.993808+00:00"
   }

9. Create and monitor the repository mirror Job. Render and apply Job/themelio-seeder using direct container environment injection. Wait for the Job to complete by reading .status.succeeded and .status.failed separately. If it fails or times out, collect Job pod status and pod logs before cleanup.

10. Clean up the Job. Delete Job/themelio-seeder and any remaining pods labeled job-name=themelio-seeder near the end of the role after status and log collection. If the Job failed, fail the role after cleanup.

#Job-controlled repository mirroring

1. Obtain a pharos-admin impersonation token. By this point, waldur-k8s-site-agent has provisioned pharos-admin as a GitLab user. The Job calls GRA's existing PUT /users/:username endpoint to get the numeric GitLab user ID, then calls POST /users/:user_id/impersonation_tokens to obtain a short-lived token with api + read_repository scopes. This token is sufficient for authenticating git push operations over HTTPS.

   No new GRA endpoints are required. The existing POST /users/:user_id/impersonation_tokens endpoint covers all repository write operations needed by the seeder.

2. Enumerate repositories. Call the external GitLab API (SEEDER_EXTERNAL_GITLAB_URL, default https://gitlab.grnet.gr) using SEEDER_EXTERNAL_GITLAB_TOKEN to list all repositories under SEEDER_EXTERNAL_GITLAB_GROUP_PATH (default themelio/dev_gitlab_backup, read from the role defaults), recursively traversing subgroups. Keep only repositories whose full path contains pharos.

   For example, skip:

   https://gitlab.grnet.gr/themelio/dev_gitlab_backup/datasets/louridas/pdf-demo-input

   Include:

   https://gitlab.grnet.gr/themelio/dev_gitlab_backup/datasets/pharos/pdf-demo-input

3. Idempotency check. For each repository, query the internal GitLab API (using the pharos-admin impersonation token) to compare existing branch HEADs. If all remote branches are already present and up to date, skip the repository.

4. Clone and push. For repositories that are missing or out of date:

   • Clone the repository (including LFS objects) from the external GitLab into /workspace using SEEDER_EXTERNAL_GITLAB_TOKEN.
   • Push all branches, tags, and LFS objects to the target path in the internal GitLab using the pharos-admin impersonation token. All seeded repositories are created as public.
   • Delete the local clone from /workspace before moving to the next repository.

#2. GitLab Root Actor Role

The gitlab-root-actor requires no modifications for the seeding workflow. The seeder uses the existing endpoint:

• POST /users/:user_id/impersonation_tokens — creates a short-lived token for pharos-admin with api + read_repository scopes, which is sufficient to authenticate git push over HTTPS to the internal GitLab instance.

The seeder never holds the internal GitLab root token. It only holds:

• The external GitLab token (for cloning from the external instance)
• The short-lived pharos-admin impersonation token (for pushing to the internal instance)

#3. Ansible Orchestration (seed-initial-admin-data role)

Seeding tasks are implemented in the dedicated seed-initial-admin-data Ansible role. The role is idempotent and can be run repeatedly.

#New Ansible tasks

The role implements the following tasks:

#New Ansible variables (defaults/main.yml)

# --- Seeder: infrastructure / Job config ---
themelio_namespace: "themelio"
themelio_seeder_image: "alpine:latest"
themelio_seeder_job_name: "themelio-seeder"
themelio_seeder_ephemeral_storage: "10Gi"
themelio_seeder_no_log: true
themelio_seeder_job_timeout_seconds: 3600

# Optional placement inherited from the platform roles.
management_node_selector: {}
management_node_tolerations: []

# --- Seeder: external GitLab (source repositories) ---
themelio_seeder_external_gitlab_url: "https://gitlab.grnet.gr"
# PAT is read from the local environment, validated, and cached by Ansible for this run.
themelio_seeder_external_gitlab_token_env: "SEEDER_EXTERNAL_GITLAB_TOKEN"
themelio_seeder_external_gitlab_token: "{{ lookup('ansible.builtin.env', themelio_seeder_external_gitlab_token_env) }}"
themelio_seeder_external_gitlab_group_path: "themelio/dev_gitlab_backup"
themelio_seeder_external_gitlab_path_filter: "pharos"

# --- Seeder: Waldur exec access ---
themelio_seeder_waldur_namespace: "waldur"
themelio_seeder_waldur_exec_deployment: "waldur-mastermind-worker"

# --- Seeder: Waldur organization ---
themelio_seeder_waldur_organization_name: "pharos"
themelio_seeder_waldur_organization_uuid: ""

# --- Seeder: Waldur admin user ---
themelio_seeder_waldur_admin_username: "pharos-admin"
themelio_seeder_waldur_admin_email: "pharos-admin@pharos.grnet.gr"
themelio_seeder_waldur_admin_first_name: "Pharos"
themelio_seeder_waldur_admin_last_name: "Admin"
themelio_seeder_waldur_admin_password: "{{ vault_themelio_seeder_waldur_admin_password | default('') }}"

# --- Seeder: Waldur project ---
themelio_seeder_waldur_project_name: "pharos"
themelio_seeder_waldur_project_slug: "pharos"

# --- Seeder: Waldur order (resource allocation) ---
# The offering UUID is resolved from the shared `waldur_offering_name` inventory variable.
themelio_seeder_waldur_allocation_name: "themelio-seed-allocation"
themelio_seeder_waldur_allocation_description: "Automated Themelio seeding allocation"
# Empty = no expiry (infinite); Waldur treats null/absent termination_date as no end date.
themelio_seeder_waldur_termination_date: ""
themelio_seeder_waldur_cpu: 10
themelio_seeder_waldur_memory_gb: 12
themelio_seeder_waldur_storage_gb: 20

# --- Seeder: GitLab Root Actor / internal GitLab ---
themelio_seeder_gitlab_root_actor_url: "{{ waldur_site_agent_cloud_gitlab_root_actor_url }}"
themelio_seeder_internal_gitlab_url: "https://{{ dns.gitlab }}"
themelio_seeder_gitlab_admin_username: "pharos-admin"
themelio_seeder_gitlab_admin_email: "pharos-admin@pharos.grnet.gr"
themelio_seeder_gitlab_admin_name: "Pharos Admin"
themelio_seeder_gitlab_admin_extern_uid: "pharos-admin"

# --- Seeder: polling / readiness ---
themelio_seeder_site_agent_namespace: "waldur-site-agent-cloud"
themelio_seeder_site_agent_order_process_app: "waldur-agent-cloud-order-process"
themelio_seeder_poll_timeout_seconds: 300
themelio_seeder_poll_interval_seconds: 10

#External GitLab token

Before running the role, the operator creates a GRNET GitLab personal access token and exports it locally as SEEDER_EXTERNAL_GITLAB_TOKEN. Ansible reads that local environment variable, validates it against the external GitLab API, and caches the token in an Ansible fact for the current run. If the environment variable is missing, empty, or the validation request indicates the token is expired, the role terminates with a clear error before creating the Kubernetes Job.

The validated token is passed directly into the rendered K8s Job container environment as SEEDER_EXTERNAL_GITLAB_TOKEN. It is never committed to vault or read from a file. Because the rendered Job contains the token, all Ansible tasks that template, apply, or display the Job manifest must use no_log.

#Vault variables

The role requires one vault variable:

vault_themelio_seeder_waldur_admin_password: "themelioisnotpharos"

Waldur operations are performed through kubectl exec into the Waldur application pod. The external GitLab token is supplied only through the operator's local SEEDER_EXTERNAL_GITLAB_TOKEN environment variable and cached as an Ansible fact for the current run.

#New templates