> For AI agents: the complete documentation index is available at /llms.txt, the full documentation bundle is available at /llms-full.txt. > **Themelio compatibility reference.** This page describes the retained Themelio implementation or its historical design. Check the [parallel-operation guide](/compatibility/index.md) before applying it to the current platform. # DD-0006: Automated Seeding Strategy โ€” Implementation **Related ADR:** [ADR-0006: Automated Data Seeding Strategy](/adr/0006-automated-seeding-strategy.md) This document provides the full technical specification for implementing the automated seeding strategy described in ADR-0006. It is intended to be followed by an engineer or an automated agent. ## 1. Kubernetes Seeder Job A single-run Kubernetes Job deployed by Ansible. The Job is limited to Git repository mirroring; it does not run Waldur orchestration, does not poll site-agent logs, and does not need Kubernetes API access. ### Container design - **Base image:** `alpine:latest` (no pre-installed tools required) - **Main container:** installs required tools at startup with `apk add --no-cache bash git git-lfs curl jq`, then runs the embedded repository mirroring script. - **Ephemeral storage:** an `emptyDir` volume at `/workspace` (default `sizeLimit: 10Gi`) is used for cloning repositories. Each repository directory is deleted after pushing to keep peak usage bounded. - **Script delivery:** Ansible renders `seeder-job.yml.j2`, which produces a complete Kubernetes Job manifest with the bash script embedded directly in `.spec.template.spec.containers[0].command` and runtime values passed through `.spec.template.spec.containers[0].env`. There is no ConfigMap or separate seeder credentials Secret โ€” the final Kubernetes resource contains the script and container environment. - **Kubernetes access:** The Job sets `automountServiceAccountToken: false` and does not define a custom ServiceAccount, Role, or RoleBinding. ### Workflow The workflow is split between the Ansible role and the Kubernetes Job. #### Ansible-controlled Waldur orchestration 1. **Reject a stale seeder Job.** Before starting, check whether `Job/themelio-seeder` already exists in the `themelio` namespace. If it exists, fail clearly and leave the existing Job untouched. 2. **Validate the external GitLab token.** Read `SEEDER_EXTERNAL_GITLAB_TOKEN` from the local environment of the operator running Ansible. Validate it with `GET https://gitlab.grnet.gr/api/v4/user`. If it is missing, invalid, or expired, fail before creating any Kubernetes Job. 3. **Run Waldur operations through `kubectl exec`.** The role uses `kubectl -n waldur exec -i deploy/waldur-mastermind-worker -- waldur shell` so Waldur-side operations run inside the application context. 4. **Validate the Waldur organization exists.** Look up an organization named `pharos`. If it exists, retrieve and store its UUID for use in subsequent steps. If it does not exist, fail with: `organization does not exist, run waldur-offering target first`. 5. **Ensure `pharos-admin` exists in Waldur.** Create or update the user if needed. This user is a Waldur staff user, has a password from vault, and is granted access to the seeded project. 6. **Ensure the Waldur project exists.** Create or reuse the project and grant `pharos-admin` project-level access. The project name and project slug are both `pharos`. 7. **Place or reuse the order.** Look up the Cloud offering by the shared `waldur_offering_name` inventory variable and use the resulting UUID to initiate or reuse an order. The role only reuses an existing order/resource for the configured allocation name if the order is not a terminate order and the linked resource is not terminated, terminating, erred, or deleted. Terminated resources and terminate orders are ignored so a fresh create order can be placed. The order uses these values: | Field | Description | | -------------------- | ------------------------------------------------------------------------ | | **ORGANIZATION** | Waldur organization (`pharos`) | | **PROJECT** | Waldur project name and slug (`pharos`) | | **RESOURCES** | Resource allocations: 10 CPU, 12 GB memory, 20 GB storage | | **ALLOCATION NAME** | Human-readable name for the order | | **DESCRIPTION** | Free-text description of the allocation | | **TERMINATION DATE** | Configurable end date for the resource allocation; empty means no expiry | The role creates the order through Waldur's in-process DRF API client, using compact UUID hyperlinks. It obtains the plan URL from the serialized `marketplace-public-offerings` response instead of constructing the plan URL by hand, sets `accepting_terms_of_service: true`, and attempts `approve_by_provider` and `approve_by_consumer` after order creation. The returned order ID is cached as an Ansible fact for the current run. Subsequent runs inspect existing Waldur resources and orders instead of relying on stored local state. This order triggers the `waldur-k8s-site-agent` to begin provisioning GitLab subgroups, users, and the Kubernetes namespace with resource quotas. 8. **Wait for `waldur-k8s-site-agent` provisioning.** After placing a new order, provisioning is handled asynchronously by `waldur-k8s-site-agent`. The agent creates GitLab subgroups and users. The role resolves the current pod for the `waldur-agent-cloud-order-process` deployment and scans that pod's logs for the returned order ID until processing completes. It searches for both compact and hyphenated UUID forms and treats the order as complete only when the matching logs contain `The order has been successfully processed`. If the role reuses an existing complete order with a non-terminated resource, this polling step is skipped. The agent emits structured JSON log events. Example events: ```jsonl { "event": "Processing order themelio-seed-allocation (d08dd31f-8ae7-483b-8d76-c40df6cfb9a3) type Create, state executing", "level": "info", "logger": "waldur_site_agent.backend", "timestamp": "2026-05-29T16:32:37.250450+00:00" } { "event": "The order d08dd31f-8ae7-483b-8d76-c40df6cfb9a3 has been successfully processed", "level": "info", "logger": "waldur_site_agent.backend", "timestamp": "2026-05-29T16:32:42.993808+00:00" } ``` 9. **Create and monitor the repository mirror Job.** Render and apply `Job/themelio-seeder` using direct container environment injection. Wait for the Job to complete by reading `.status.succeeded` and `.status.failed` separately. If it fails or times out, collect Job pod status and pod logs before cleanup. 10. **Clean up the Job.** Delete `Job/themelio-seeder` and any remaining pods labeled `job-name=themelio-seeder` near the end of the role after status and log collection. If the Job failed, fail the role after cleanup. #### Job-controlled repository mirroring 1. **Obtain a `pharos-admin` impersonation token.** By this point, `waldur-k8s-site-agent` has provisioned `pharos-admin` as a GitLab user. The Job calls GRA's existing `PUT /users/:username` endpoint to get the numeric GitLab user ID, then calls `POST /users/:user_id/impersonation_tokens` to obtain a short-lived token with `api` + `read_repository` scopes. This token is sufficient for authenticating git push operations over HTTPS. > **No new GRA endpoints are required.** The existing `POST /users/:user_id/impersonation_tokens` endpoint covers all repository write operations needed by the seeder. 2. **Enumerate repositories.** Call the external GitLab API (`SEEDER_EXTERNAL_GITLAB_URL`, default `https://gitlab.grnet.gr`) using `SEEDER_EXTERNAL_GITLAB_TOKEN` to list all repositories under `SEEDER_EXTERNAL_GITLAB_GROUP_PATH` (default `themelio/dev_gitlab_backup`, read from the role defaults), recursively traversing subgroups. Keep only repositories whose full path contains `pharos`. For example, skip: ```text https://gitlab.grnet.gr/themelio/dev_gitlab_backup/datasets/louridas/pdf-demo-input ``` Include: ```text https://gitlab.grnet.gr/themelio/dev_gitlab_backup/datasets/pharos/pdf-demo-input ``` 3. **Idempotency check.** For each repository, query the internal GitLab API (using the `pharos-admin` impersonation token) to compare existing branch HEADs. If all remote branches are already present and up to date, skip the repository. 4. **Clone and push.** For repositories that are missing or out of date: - Clone the repository (including LFS objects) from the external GitLab into `/workspace` using `SEEDER_EXTERNAL_GITLAB_TOKEN`. - Push all branches, tags, and LFS objects to the target path in the internal GitLab using the `pharos-admin` impersonation token. All seeded repositories are created as public. - Delete the local clone from `/workspace` before moving to the next repository. *** ## 2. GitLab Root Actor Role The `gitlab-root-actor` requires **no modifications** for the seeding workflow. The seeder uses the existing endpoint: - `POST /users/:user_id/impersonation_tokens` โ€” creates a short-lived token for `pharos-admin` with `api` + `read_repository` scopes, which is sufficient to authenticate git push over HTTPS to the internal GitLab instance. The seeder never holds the internal GitLab root token. It only holds: - The external GitLab token (for cloning from the external instance) - The short-lived `pharos-admin` impersonation token (for pushing to the internal instance) *** ## 3. Ansible Orchestration (`seed-initial-admin-data` role) Seeding tasks are implemented in the dedicated `seed-initial-admin-data` Ansible role. The role is idempotent and can be run repeatedly. ### New Ansible tasks The role implements the following tasks: | # | Task | Inputs | Effect | | - | -------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 0 | **Reject existing seeder Job** | `themelio_namespace`, `themelio_seeder_job_name` | Fails clearly if `Job/themelio-seeder` already exists at startup; the role does not delete or overwrite a pre-existing Job. | | 1 | **Validate and cache external GitLab token** | Local `SEEDER_EXTERNAL_GITLAB_TOKEN` environment variable, `themelio_seeder_external_gitlab_url` | Reads the operator-provided token from the local environment, validates it against the external GitLab API, caches it in an Ansible fact for this run, and fails if the variable is missing or the token is expired. | | 2 | **Validate seeder inputs** | Seeder defaults, shared `waldur_offering_name`, shared GitLab Root Actor URL | Fails fast with a clear error message if any required variable is missing. | | 3 | **Validate Waldur organization** | `themelio_seeder_waldur_organization_name`, `themelio_seeder_waldur_exec_deployment` | Uses `kubectl exec` into Waldur to look up the organization; fails with `organization does not exist, run waldur-offering target first` if missing; otherwise registers the organization UUID. | | 4 | **Ensure Waldur user and project** | `themelio_seeder_waldur_admin_username`, `themelio_seeder_waldur_admin_password`, `themelio_seeder_waldur_project_name`, `themelio_seeder_waldur_project_slug` | Creates or updates the `pharos-admin` Waldur staff user, sets its vault-backed password, grants project-manager access, and creates or reuses the `pharos` project. | | 5 | **Resolve Waldur cloud offering** | Shared `waldur_offering_name` | Uses `kubectl exec` into Waldur to look up the cloud offering by name; caches the returned offering UUID as an Ansible fact for the current run. | | 6 | **Create or reuse Waldur order** | Organization UUID, project UUID, offering UUID, CPU/memory/storage defaults | Reuses only matching non-terminate orders whose linked resource is not terminated, terminating, erred, or deleted; otherwise creates a new order with 10 CPU, 12 GB memory, and 20 GB storage. No GPU fields are sent. The plan URL is read from the public offering serializer, and the role attempts provider and consumer approval after order creation. | | 7 | **Monitor `waldur-k8s-site-agent`** | `waldur-site-agent-cloud` namespace, `waldur-agent-cloud-order-process` deployment pod, order ID | Resolves the current order-process pod, scans `--since=60m` logs for compact or hyphenated order UUIDs, and waits for `The order has been successfully processed`. | | 8 | **Deploy seeder Job** | `seeder-job.yml.j2` template; cached GitLab token; GitLab Root Actor URL; repository defaults; `management_node_selector`, `management_node_tolerations` | Creates `Job/themelio-seeder` in the `themelio` namespace with required `SEEDER_*` variables defined directly in the main container `env`; token-bearing tasks use `no_log`. | | 9 | **Collect Job result and clean up** | Job status, Job pod status, Job pod logs | Waits for Job completion by reading `.status.succeeded` and `.status.failed` separately, collects pod status/logs on failure, deletes `Job/themelio-seeder` and remaining Job pods, and fails the role after cleanup if the Job failed. | ### New Ansible variables (`defaults/main.yml`) ```yaml # --- Seeder: infrastructure / Job config --- themelio_namespace: "themelio" themelio_seeder_image: "alpine:latest" themelio_seeder_job_name: "themelio-seeder" themelio_seeder_ephemeral_storage: "10Gi" themelio_seeder_no_log: true themelio_seeder_job_timeout_seconds: 3600 # Optional placement inherited from the platform roles. management_node_selector: {} management_node_tolerations: [] # --- Seeder: external GitLab (source repositories) --- themelio_seeder_external_gitlab_url: "https://gitlab.grnet.gr" # PAT is read from the local environment, validated, and cached by Ansible for this run. themelio_seeder_external_gitlab_token_env: "SEEDER_EXTERNAL_GITLAB_TOKEN" themelio_seeder_external_gitlab_token: "{{ lookup('ansible.builtin.env', themelio_seeder_external_gitlab_token_env) }}" themelio_seeder_external_gitlab_group_path: "themelio/dev_gitlab_backup" themelio_seeder_external_gitlab_path_filter: "pharos" # --- Seeder: Waldur exec access --- themelio_seeder_waldur_namespace: "waldur" themelio_seeder_waldur_exec_deployment: "waldur-mastermind-worker" # --- Seeder: Waldur organization --- themelio_seeder_waldur_organization_name: "pharos" themelio_seeder_waldur_organization_uuid: "" # --- Seeder: Waldur admin user --- themelio_seeder_waldur_admin_username: "pharos-admin" themelio_seeder_waldur_admin_email: "pharos-admin@pharos.grnet.gr" themelio_seeder_waldur_admin_first_name: "Pharos" themelio_seeder_waldur_admin_last_name: "Admin" themelio_seeder_waldur_admin_password: "{{ vault_themelio_seeder_waldur_admin_password | default('') }}" # --- Seeder: Waldur project --- themelio_seeder_waldur_project_name: "pharos" themelio_seeder_waldur_project_slug: "pharos" # --- Seeder: Waldur order (resource allocation) --- # The offering UUID is resolved from the shared `waldur_offering_name` inventory variable. themelio_seeder_waldur_allocation_name: "themelio-seed-allocation" themelio_seeder_waldur_allocation_description: "Automated Themelio seeding allocation" # Empty = no expiry (infinite); Waldur treats null/absent termination_date as no end date. themelio_seeder_waldur_termination_date: "" themelio_seeder_waldur_cpu: 10 themelio_seeder_waldur_memory_gb: 12 themelio_seeder_waldur_storage_gb: 20 # --- Seeder: GitLab Root Actor / internal GitLab --- themelio_seeder_gitlab_root_actor_url: "{{ waldur_site_agent_cloud_gitlab_root_actor_url }}" themelio_seeder_internal_gitlab_url: "https://{{ dns.gitlab }}" themelio_seeder_gitlab_admin_username: "pharos-admin" themelio_seeder_gitlab_admin_email: "pharos-admin@pharos.grnet.gr" themelio_seeder_gitlab_admin_name: "Pharos Admin" themelio_seeder_gitlab_admin_extern_uid: "pharos-admin" # --- Seeder: polling / readiness --- themelio_seeder_site_agent_namespace: "waldur-site-agent-cloud" themelio_seeder_site_agent_order_process_app: "waldur-agent-cloud-order-process" themelio_seeder_poll_timeout_seconds: 300 themelio_seeder_poll_interval_seconds: 10 ``` ### External GitLab token Before running the role, the operator creates a GRNET GitLab personal access token and exports it locally as `SEEDER_EXTERNAL_GITLAB_TOKEN`. Ansible reads that local environment variable, validates it against the external GitLab API, and caches the token in an Ansible fact for the current run. If the environment variable is missing, empty, or the validation request indicates the token is expired, the role terminates with a clear error before creating the Kubernetes Job. The validated token is passed directly into the rendered K8s Job container environment as `SEEDER_EXTERNAL_GITLAB_TOKEN`. It is never committed to vault or read from a file. Because the rendered Job contains the token, all Ansible tasks that template, apply, or display the Job manifest must use `no_log`. ### Vault variables The role requires one vault variable: ```yaml vault_themelio_seeder_waldur_admin_password: "themelioisnotpharos" ``` Waldur operations are performed through `kubectl exec` into the Waldur application pod. The external GitLab token is supplied only through the operator's local `SEEDER_EXTERNAL_GITLAB_TOKEN` environment variable and cached as an Ansible fact for the current run. ### New templates | Template | Produces | | ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | | `seeder-job.yml.j2` | `Job/themelio-seeder` with a single main container, workspace volume, `automountServiceAccountToken: false`, direct container `env`, and bash script inline |
Page source Source snapshot ยท Revision `f928834c8`.