Skip to content

Selective Istio Sidecar Injection Due to AKS Constraints

Context and Problem Statement

AKS Automatic blocks NET_ADMIN and NET_RAW capabilities, which are required by Istio's istio-init container to configure iptables rules for traffic interception. This means standard Istio sidecar injection fails for any namespace where it is enabled. We need a strategy for service mesh coverage that works within AKS Automatic's capability restrictions.

Decision Drivers

  • AKS Automatic blocks NET_ADMIN/NET_RAW capabilities (no exceptions possible)
  • Istio istio-init container requires NET_ADMIN for iptables configuration
  • Data namespaces (Elasticsearch, PostgreSQL) benefit from mTLS encryption
  • RabbitMQ, ExternalDNS, and other workloads must function without mesh injection
  • Istio ambient mode (ztunnel-based, no istio-init needed) is on the AKS roadmap

Considered Options

  • Selective sidecar injection (inject where it works, skip where it doesn't)
  • No service mesh at all
  • Linkerd (no NET_ADMIN requirement)
  • Wait for Istio ambient mode before deploying mesh

Decision Outcome

Chosen option: "Selective sidecar injection now, with ambient mode as future aspiration", because it provides mTLS where possible today while maintaining forward compatibility with ambient mode when AKS supports it.

Implementation

Both namespaces have Istio sidecar injection enabled at the namespace level, with pod-level opt-outs for workloads that conflict with AKS Automatic's capability restrictions (see ADR-0017):

  • platform: Istio sidecar injection enabled (istio-injection: enabled). STRICT mTLS enforced via PeerAuthentication. Specific pods that require NET_ADMIN/NET_RAW capabilities (e.g., RabbitMQ) opt out at the pod level via sidecar.istio.io/inject: "false" annotation. These workloads rely on application-layer security (password auth, ECK self-signed TLS for Elasticsearch).
  • osdu: Istio sidecar injection enabled (istio-injection: enabled). All OSDU services run with Istio sidecars and STRICT mTLS via PeerAuthentication.

Consequences

  • Good, because both namespaces have STRICT mTLS via PeerAuthentication and sidecar injection by default
  • Good, because pod-level opt-out (sidecar.istio.io/inject: "false") provides fine-grained control without sacrificing namespace-wide mesh coverage
  • Good, because Istio ingress gateway works without sidecar injection (uses Gateway API)
  • Good, because forward-compatible with ambient mode — when available, pod-level opt-outs become unnecessary
  • Bad, because pods that opt out of injection (RabbitMQ, etc.) rely on application-layer security rather than mesh mTLS
  • Bad, because istio-init failures on opted-out pods can be confusing during debugging if the annotation is missing