Adopting Ambient Istio

What is a Service Mesh

Abstracting network complexity from the Application layer

Core pillars: Security Observability Traffic management

Istio is the industry standard, but know to be heavy to deploy

Sidecar Model

Traditionally you run a sidecar container.

Problems:

  • Cost (each pod gets a sidecar)
  • Intercepts all inbound and outbound traffic
  • New versions are deployed on pod restart -> leads to outdated versions

Shift to Ambient

In Ambient mode, the data plane is sidecarless.

Core concept: Decouple Layer 4 from Layer 7.

Workloads talk to each other via ztunnels

Gateway API

If you’re using Ambient mode, Istio suggests using Gateway API.

Aside from that, the speaker remarks that the Istio CRDs are uh… shitty

Is this a problem? Not really, but Gateway API is early stages.

Evaluating Ambient mode

The Good

  • Cost is down (reduction of 30-60%)
  • Zero-friction operations: Istio upgrades no longer require restarting business apps for new Sidecar versions.
  • Predictable infrastructure footprint: ztunnel scales with nodes, not with pods
  • Simplified Metrics: sidecar is also a monitorable component

The Bad

  • Layer 7 features require a Waypoint, which needs to be configured using Gateway API, some vendors support Layer 7 over ztunnel, but not OSS
  • Authz complexity skyrockets when mixing Layer 4 and Layer 7 (Split Brain)
  • All or nothing
    • ztunnel transparently intercepts ALL traffic on the nodes
    • You lose the ability to easily filter out or exclude certain ports at the pod level
    • It’s a blackbox for traditional pod debugging

The Ugly

  • ztunnel metrics look different from traditional Envoy sidecar metrics
  • Traffic flow has changed to: App -> ztunnel -> HBONE -> ztunnel -> App
  • Your monitoring dashboards need to be adapted to this
  • Gateway API is needed, which is not very mature yet
  • Gateway API usage means using experimental features in other tooling
    • external-dns
    • cert-manager
    • argo-rollouts
    • …

Conclusion

Is Ambient mode worth it? Yes

But, there’s trade-offs.

Pro tip: use Layer 4 as much as possible.