Skip to main content

🔌 Cilium CNI support (Secure Containers)

FluxVM does not ship a Cilium fork or write Cilium-private BPF maps. On Cilium nodes, Secure Containers keep Cilium as the Pod CNI and hand the Pod's primary L2 endpoint (eth0 IP/MAC/routes) into the guest VM.

Network Fabric sandbox.dataplane.mode = "cilium" is a separate coexistence path for VM-edge TC policy — see ebpf-cilium.md. This document covers the containerd shim CNI L2 path.

✅ What works​

PieceBehavior
ProviderFLUXVM_CONTAINER_CNI_PROVIDER=auto|cilium|generic (default auto)
Detectionauto checks cilium.sock, /opt/cni/bin/cilium-cni, /etc/cni/net.d/*cilium*, and pod-side cilium* ifaces
Primary ifacePrefers configured iface (default eth0); falls back to first routable non-Multus iface
MultusnetN secondaries are prepared as extra host bridges and attached as guest NICs (network.extra at create, or POST /v1/vms/{id}/hotplug/nic after a warm-pool claim)
Dual-stackIPv4 + IPv6 addresses/routes on the primary iface (Set 6)
Host TCCilium programs stay on the host lxc* peer; FluxVM bridges the pod-side endpoint

⚠️ What does not work (honesty bounds)​

  • Non-netN extra interfaces still fail closed when FLUXVM_CONTAINER_CNI_STRICT_MULTI_INTERFACE=1.
  • Writing Cilium identity / ipcache / endpoints maps.
  • Replacing Cilium NetworkPolicy with FluxVM as the Pod CNI.
  • Claiming Hubble attribution for every guest packet without CEP enrich (hubble-lite.md).

🛠️ Operator setup​

  1. Install Secure Containers (scripts/install-secure-containers.sh) and merge deploy/containerd/fluxvm-runtime.toml.
  2. On Cilium nodes, export shim env (or use configs/cilium-cni.toml as the Network Fabric coexistence fragment + the env comments as SoT):
export FLUXVM_CONTAINER_CNI=1
export FLUXVM_CONTAINER_CNI_PROVIDER=auto   # or force cilium
export FLUXVM_CONTAINER_CNI_INTERFACE=eth0
  1. Optional VM-edge Fabric coexistence:
sudo ./scripts/enable-network-fabric-ga.sh --cilium --restart

  1. Deploy RuntimeClass / DaemonSet mounts from deploy/k8s/cilium/.

  2. Evidence:

./scripts/evidence-cilium-cni.sh
# live (optional): FLUXVM_CILIUM_CNI_LIVE=1 ./scripts/evidence-cilium-cni.sh

📡 Packet path (primary)​

lxc* ⇄ Pod netns eth0 (CNI-assigned IP; the guest takes over IP/MAC/routes)
  → in-Pod bridge → second veth pair → host bridge → guest TAP → guest virtio-net
Host lxc* peer keeps Cilium TC/eBPF; FluxVM never writes Cilium maps.

That bridge chain is the fallback. By default (FLUXVM_CONTAINER_CNI_DATAPATH=auto, a veth Pod, no Multus secondaries, kernel ≥ 5.10, an ebpf/cilium daemon dataplane) a TC redirect replaces it; FLUXVM_CONTAINER_CNI_DATAPATH=bridge forces the chain — see direct-datapath.md.

📚 Related​