Cilium Service Mesh: When You Don’t Need Sidecars

Cilium^[1] started as an eBPF-based CNI and evolved into a full alternative to traditional service meshes, no sidecars needed. Its architecture leans on eBPF to handle policy, observability, and encryption right in the kernel, with no per-pod proxy. For large clusters the resource savings are considerable.

Istio answered with Ambient Mode, a similar idea. This article compares the sidecarless approach and when to pick each one.

Key takeaways

• Cilium eliminates sidecar overhead using eBPF in kernel: ~5 GB RAM total overhead in a 100-node cluster vs ~100 GB for classic Istio.

• Per-node WireGuard encryption is less granular than per-service mTLS but much more efficient.

• For L7 policies (HTTP verbs, paths), Cilium starts an Envoy per node on-demand, not per pod.

• Hubble is the integrated observability layer: flows, dependency maps, and policy verdicts without additional tools.

• Cilium has more ground in sidecarless than Istio Ambient (GA 2023 vs GA 2024).

The sidecar problem

The traditional sidecar model (Linkerd, classic Istio) has real costs:

• One Envoy/linkerd-proxy per pod.

• Resource overhead: 50-200 MB RAM + CPU per pod.

• Additional latency: 2-5 ms round-trip.

• Operational complexity: more processes, lifecycle management.

In clusters with thousands of pods, multiplied by sidecar, the numbers add up fast. The comparison with Linkerd, which already uses a lighter Rust proxy, is still worth making: Linkerd runs about 10 GB of overhead across 100 nodes, Cilium about 5 GB. I go deeper on that comparison in service mesh: Istio, Linkerd, and the Cilium option.

Cilium’s approach

Cilium replaces sidecar proxies with:

• eBPF in kernel for simple policy and encryption.

• Centralised Envoy per node for complex L7 features (only where used).

• Hubble for native observability.

• CNI integration: Cilium is both CNI and service mesh in one piece.

The result: comparable features with less overhead.

mTLS with eBPF

Cilium encrypts inter-node traffic with WireGuard (simple and fast) or IPsec (more compatible), without sidecar injection:

apiVersion: cilium.io/v2
kind: CiliumClusterwideNetworkPolicy
metadata:
  name: enforce-encryption
spec:
  endpointSelector: {}
  ingress:
    - fromEntities:
        - cluster

WireGuard runs per node, not per pod: it is less granular than per-service mTLS, but a lot more efficient. For strict multi-tenant setups that need per-pod identity, Istio Ambient with per-identity mTLS is the better fit.

L7 policies

For traffic that needs L7 handling (HTTP verbs, paths), Cilium spins up an Envoy per node (not per pod) on demand:

apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
spec:
  endpointSelector:
    matchLabels:
      app: api
  ingress:
    - fromEndpoints:
        - matchLabels:
            app: frontend
      toPorts:
        - ports:
            - port: "80"
              protocol: TCP
          rules:
            http:
              - method: "GET"
                path: "/api/v1/users"

Hubble: integrated observability

Hubble gives you:

• Detailed per-connection flow logs.

• Service dependency maps (who talks to whom).

• Policy verdicts (why a request was allowed/denied).

• Prometheus and Grafana integration.

Functional equivalent to Kiali + linkerd-viz but integrated without additional tools.

Cilium vs Istio Ambient

Cilium has more ground in sidecarless. Istio Ambient is newer, but it inherits Istio’s mature ecosystem.

Resource comparisons

Orientation benchmark (cluster 100 nodes, 1000 pods):

These are ballpark numbers that depend heavily on the specific configuration.

When to choose Cilium

Good fits:

• Large clusters (more than 500 pods) where sidecar overhead matters.

• Teams with eBPF experience or willing to invest.

• Greenfield Kubernetes without legacy CNI.

• L7 policy needs with high throughput.

• Multi-cluster with advanced connectivity requirements.

When not to choose Cilium

• Small cluster where sidecars aren’t a real problem.

• Already running Istio with complex features: migrating won’t pay off.

• Team without low-level networking experience.

• Fine per-pod identity requirements (prefer Istio Ambient).

• Switching CNIs on a running cluster is a disruptive project in its own right.

Commercial ecosystem

Isovalent^[2] (the company behind Cilium) was acquired by Cisco, announced in December 2023 and closed in April 2024. That validates the project at the enterprise level, but it also opens the door to the vendor pushing its own commercial agenda. The open-source alternative keeps working without any dependency on Cisco.

Conclusion

Cilium represents a genuine service-mesh evolution: sidecarless, eBPF-native, CNI-integrated. For large clusters and technically capable teams, it offers real resource and feature advantages. It is not the right choice for everyone.

Linkerd remains valid for its simplicity, classic Istio for feature-completeness, and Istio Ambient as a sidecarless alternative with different trade-offs. The service-mesh choice in 2024 has more mature options than ever; the decision should rest on your technical context and your team, not on the trend of the moment.

¿Lees en español? Versión completa: Cilium Service Mesh: cuando no necesitas sidecars.

Frequently asked questions

How much memory does Cilium save compared with a sidecar-based service mesh?

In an orientation benchmark of 100 nodes and 1000 pods, Cilium + CNI adds about 5 GB of RAM total overhead. Classic Istio with sidecars adds about 100 GB, Linkerd about 10 GB and Istio Ambient about 15 GB. The reason is that each sidecar costs 50-200 MB RAM plus CPU per pod and adds 2-5 ms of round-trip latency; these are ballpark figures that depend heavily on the configuration.

Does Cilium provide per-service mTLS like Istio?

Not exactly. Cilium encrypts inter-node traffic with WireGuard (or IPsec, which is more compatible) and does so per node, not per pod. It is less granular than per-service mTLS but much more efficient and needs no sidecar injection. If you need strict multi-tenant setups with per-pod identity, Istio Ambient with per-identity mTLS in ztunnel is the better fit.

How does Cilium enforce L7 policies (HTTP paths and verbs) without sidecars?

By spinning up one Envoy per node on demand, only where L7 handling is used, instead of a proxy per pod. The policy is declared in a CiliumNetworkPolicy with http rules specifying method and path, for example GET on /api/v1/users from pods labelled app: frontend. The remaining L4 traffic and encryption are handled directly in the kernel with eBPF.

Sources

1. Cilium
2. Isovalent