#ADR-0025: Scoped Istio Authorization Policies

Status: Draft

Date: 2026-07-16

Deciders: TBD

Related: ADR-0024 Baseline L3/L4 Network Segmentation (referenced in the original record; absent from the archived tree)

#Context

Themelio exposes platform applications through Envoy ingress gateways in istio-system. Several gateway AuthorizationPolicy resources use action: ALLOW with an empty rule. That rule matches every request and does not restrict access.

Application workloads are not generally enrolled in the Istio mesh. The repository has no general sidecar injection or mesh-wide mTLS enforcement. The current ingress gateways can enforce request policy, but most backends cannot yet authenticate a calling workload through Istio.

We want to protect both entry points and selected internal calls. Ingress policy will restrict configured HTTP hosts. Backend policy will use ServiceAccount-derived workload identity and mTLS so that an unapproved pod cannot call a protected service directly.

An ingress host rule checks the HTTP Host or :authority value. It is not a DNS check and does not prove which DNS name the client resolved.

#Requirements

1. In enforced, ingress gateways must accept only their configured HTTP hosts.
2. In enforced, selected internal services must authenticate callers by workload identity over mTLS.
3. Every workload trusted by identity must use a dedicated Kubernetes ServiceAccount.
4. Authorization must start with caller identity and destination port.
5. HTTP method and path checks will be added only where they are stable and useful.
6. Platform automation must own mesh enrollment and security policy.
7. The design must avoid adding a sidecar to every workload by default.

#Considered Options

#Option A: Use Cilium service mesh

Cilium provides service mesh and L7 policy capabilities. Its mutual authentication feature, which is required for authenticated workload identity in this ADR, is currently documented as beta and incomplete.

Cilium is not selected for service-to-service authorization in this ADR. This does not mean that all Cilium networking or service mesh features are beta. The networking provider discussion remains separate in ADR-0024.

#Option B: Continue with Istio sidecars

Inject an Istio sidecar into each selected caller and destination. Enforce strict mTLS at protected destinations and allow requests by ServiceAccount identity and destination port. A sidecar can also enforce HTTP methods and paths.

This is mature and suitable for a small pilot. Each participating pod carries its own proxy, resource use, configuration, and upgrade work. The cost grows with every pod and becomes less attractive if many internal flows are added later.

Calico requires connect-time load balancing to be disabled so service mesh sidecars can see Kubernetes Service addresses. This option can use the Calico eBPF dataplane, but not its connect-time load-balancing optimization.

#Option C: Use Istio ambient mode

Install the Istio ambient components and enroll only the selected workloads. A ztunnel on each node provides transparent mTLS, ServiceAccount-derived identity, and L4 authorization without an application sidecar.

Waypoints are not required for mTLS. They are added only where L7 policy or routing is needed, such as HTTP methods, paths, or headers. Existing ingress gateways can remain in place.

This reduces per-pod proxy overhead and makes later enrollment easier. It adds ztunnel and optional waypoint operations. Ambient L7 AuthorizationPolicy is still beta, so the first internal policies should remain at identity and port level.

This option is not selected. Kubeflow Profile and other HTTP authorization flows would require an additional waypoint and policy migration, while scoped sidecars preserve the existing enforcement model.

#Decision

We propose Option B, scoped Istio sidecar injection. Only the callers and destinations named by this ADR receive a sidecar; namespace-wide injection is not introduced for Themelio or Waldur. The ADR remains Draft until the sidecar canary proves the current gateways, VirtualService resources, Calico networking, application startup, and operational tooling.

Istio CNI remains installed so sidecars do not require a privileged istio-init container. Calico remains on its standard dataplane during the pilot. If Calico eBPF is enabled later, connect-time load balancing must remain disabled so the sidecars can observe Kubernetes Service addresses.

See also: DD-0025: Scoped Istio Authorization Policies — Implementation

#Security profiles

ADR-0025 enrollment and enforcement are controlled by security_profile:

Local and dev use open; staging and production use enforced after their canary gates pass. permissive is the intermediate rollout and rollback state used to verify sidecar mTLS and workload identities before restrictive policies are applied. Dedicated ServiceAccounts and sidecar-capable Istio CNI infrastructure remain installed in every profile. Existing application authentication, user RBAC, service credentials, and CUSTOM external authorization are independent controls and are not disabled by open.

#Ingress policy

In enforced, each ingress gateway will have a platform-owned ALLOW policy that selects only that gateway workload. It will allow the hosts configured on its Gateway and VirtualService, including the form with a port when clients may send one. Once an ALLOW policy selects a gateway, unmatched requests are denied at that gateway. The ADR-0025 native ALLOW policy is absent in open and permissive.

Host policy is defense in depth. Gateway and VirtualService resources remain the source of truth for listeners and routing. Existing external authorization integrations will remain in place.

In the current deployment, public HTTPS terminates at Nginx on the bastion. Nginx forwards plain HTTP over the private network to the gateway's HTTP NodePort and preserves the Host header. The gateway can therefore check the HTTP host, method, and path. For backend workloads added to the mesh, Istio mTLS protects traffic from the ingress gateway onward.

We will not add one root namespace deny-all policy at first. A scoped ALLOW policy already denies unmatched requests at the selected gateway and limits the impact of a mistake.

#Internal workload policy

The first scoped sidecar injection will cover these identities and flows:

• the web ingress gateway to web
• web to k8s-root-actor
• web to gitlab-root-actor
• the Waldur order process to gitlab-root-actor
• the Waldur membership sync process to gitlab-root-actor
• the seeder to gitlab-root-actor if that call remains

The directional communication matrix for the selected application services is:

Rows are callers and columns are destinations. A ✓ means ADR-0025 defines and tests that directional connection; X means it does not. Cells where the caller and destination are the same service are intentionally empty. A reverse call requires its own ✓. The ingress-gateway → Web and conditional seeder → gitlab-root-actor infrastructure flows remain in the preceding list because they are not application services represented by this matrix. Application credentials and human authorization remain required after the workload-level connection is admitted.

Solid edges are allowed flows enforced by STRICT mTLS and ServiceAccount identity; dashed edges are denied or conditional. web, k8s-root-actor, and gitlab-root-actor are the protected destinations.

k8s-root-actor additionally generates one scoped <name>-service-gateway policy per tenant deployment and inference service, replacing the former namespace-wide allow-all:

Each generated policy admits only the Themelio service-gateway principal on the workload port, ns-owner-access-istio retains same-namespace access. Tenants cannot manage these policies ownership stays with platform automation.

k8s-root-actor already has a dedicated ServiceAccount. web, gitlab-root-actor, the web ingress gateway, and the seeder will receive dedicated ServiceAccounts before their identities are trusted. The Waldur order and membership processes share the dedicated waldur-site-agent-cloud-gitlab-sync ServiceAccount because they have the same trust level. Report and component-loading retain their existing identity and remain denied from gitlab-root-actor. These identities will be split further if their allowed flows diverge.

The web and ArgoCD gateways currently share the istio-ingress ServiceAccount. The GitLab, Waldur, image registry, Grafana, Prometheus, and Ceph dashboard gateways currently use the istiod ServiceAccount. Each gateway must receive its own ServiceAccount before a backend AuthorizationPolicy trusts that gateway identity.

Every allowed caller and protected destination must be sidecar-injected before strict mTLS is enforced. Selected destinations will then enforce strict mTLS. Their initial AuthorizationPolicy rules allow only documented ServiceAccount identities on documented destination ports. Sidecars can enforce HTTP methods, paths, and headers, but this ADR intentionally starts with identity and port rules only.

PostgreSQL and Valkey are not included in the first mesh enrollment. ADR-0024 NetworkPolicy, database or cache authentication, and application TLS will protect them. They can be reconsidered if there is a clear need for mesh identity or telemetry.

Application authentication and user RBAC remain required. Istio workload identity answers which workload made a request, not which human user is authorized to perform an action.

#Ownership and rollout

AuthorizationPolicy, PeerAuthentication, RequestAuthentication, and sidecar-injection labels will be owned through platform GitOps or Ansible. Tenant roles will not be able to manage them or change the labels and ServiceAccounts used for policy identity. Existing NetworkPolicy application ports remain required; no Ambient HBONE exception is introduced.

Rollout will first replace empty ingress rules one gateway at a time. It will then assign the dedicated ServiceAccounts and run a sidecar canary for web to k8s-root-actor. The remaining internal flows will receive sidecars only after positive and negative tests pass.

Each rollout must test an accepted request, an unknown host or caller, direct backend access from an unapproved pod, and the service's normal browser and API flows. istioctl analyze, Envoy sidecar logs/configuration, and policy metrics will be checked before promotion.

#Observability

Istio monitoring will show where a request was rejected. Ingress Envoy access logs and metrics cover gateway and host policy decisions. Application Envoy sidecar logs and metrics cover mTLS identity and backend authorization failures.

These signals will be sent to the existing Prometheus, Grafana, and Loki stack. Dashboards and alerts will distinguish gateway rejection, mTLS failure, workload identity denial, NetworkPolicy denial, and application failure where the available telemetry allows it. NetworkPolicy denials will use the flow visibility defined in ADR-0024.

#Validation status

ADR-0025 is validated in staging one check at a time using the ADR-0025 validation document. A row may be changed to ✅ Passed only after its command or manual procedure has been run and its non-secret evidence has been recorded. ⏳ Pending, ⚠️ Blocked, and ❌ Failed keep this ADR in Draft. The conditional seeder row may be marked ➖ Not applicable only if the seeder no longer calls gitlab-root-actor.

The application rows are limited to regression paths affected by ADR-0025 gateways, identities, mTLS, or observability. They are not a general platform release test suite.

#Consequences

Positive:

• Unknown HTTP hosts are rejected at the selected ingress gateway.
• Existing OAuth and external authorization integrations remain in place.
• Protected backends can reject direct calls from unapproved workloads.
• Sidecars are injected only into the callers and destinations in scope.
• Policy ownership stays with platform automation rather than tenant users.

Negative:

• A missing host or listener rule can return an unexpected 403 response.
• The bastion-to-gateway hop remains plain HTTP on the private network.
• Each selected pod carries an Envoy proxy with measurable CPU and memory cost.
• Sidecar image/configuration upgrades roll every selected workload.

Neutral:

• These policies authenticate workloads, not human users.
• NetworkPolicy remains required for L3 and L4 reachability.
• Namespace-wide sidecar injection remains out of scope.

#References

• Istio Authorization Policy
• Istio Sidecar Injection
• Istio CNI
• Istio Envoy Access Logs
• Istio Standard Metrics
• Istio Security Best Practices
• Cilium Mutual Authentication

Page source

Source snapshot · Revision f928834c8.