Skip to main content

User guide: Machine networking

How to configure Machine.spec.network and related status fields. Kairon maps these into FluxVM create/status APIs; it does not run Multus or own BPF programs. Boundary details: network-fabric.md.

Modes​

modeWhen to useKey fields
user (default)Lab / NAT without a host bridgeforwards[] for host→guest ports
tapBridged or isolated edge (Fabric/eBPF path)netns, bridge, tapName, mac
macvtapDirect MAC on a parent NICparent, macvtapMode, mac

User mode forwards​

spec:
  network:
    mode: user
    forwards:
      - hostPort: 8080
        guestPort: 80
        protocol: tcp   # tcp|udp; default tcp

FluxVM receives host_port / guest_port / protocol on the create payload, bound to 0.0.0.0 so it's reachable off-host (e.g. ssh -p 8080 <node-ip> reaching guest port 22 if forwarded). kaironctl create --forward=8080:80 (repeatable) and the dashboard's create form set this without hand-writing a manifest.

Forwards are only applied at Machine creation. Editing spec.network.forwards on an already-running Machine has no effect -- kairon-node only reads it inside the one-time FluxVM create call, never on later reconcile ticks. The same is true of spec.resources (CPU/memory) after creation: change it and nothing happens, silently, no error and no status signal. Delete and recreate the Machine to apply either change.

TAP + netns (recommended for Network Fabric)​

spec:
  network:
    mode: tap
    netns: true
    staticNetwork: true   # cloud-init static address (tap+netns only)
    tapName: tap-web      # optional; FluxVM may allocate
    mac: "52:54:00:12:34:56"
    dataplaneRequired: true
    dataplaneMode: cilium   # optional: legacy|ebpf|cilium (empty = FluxVM default)
  • netns: true — per-VM network namespace (isolation + known guest address).
  • staticNetwork: true — sets FluxVM cloud_init.static_network so the guest does not depend on DHCP.
  • dataplaneRequired: true — node agent fail-closes if eBPF/cilium attach is unhealthy.
  • dataplaneMode — requests FluxVM network.dataplane_mode on create. For a cluster-wide Cilium dataplane default, also set /etc/fluxvm.toml sandbox.dataplane.mode = "cilium" on each node (Helm network.ciliumDataplane documents this; it does not rewrite the TOML itself).
  • mac — optional. A netns: true Machine that omits it gets a stable generated MAC (52:54:00: plus three bytes of an FNV-1a hash of namespace and name), which FluxVM requires for netns networking. Must be a standard 6-octet Ethernet address (colon- or hyphen-separated hex, like the example above). With webhook.enabled set, a malformed mac is rejected immediately on kubectl apply/edit instead of only failing once kairon-node asks FluxVM to attach the NIC -- see SECURITY.md.

VM edge: anti-spoof, learn-IP, QoS​

spec:
  network:
    mode: tap
    netns: true
    mac: "52:54:00:12:34:56"
    dataplaneMode: ebpf
    antiSpoof: true
    learnIP: true
    qos:
      ingressMbps: 100   # network -> guest
      egressMbps: 50     # guest -> network
      ingressPps: 20000
      egressPps: 2000
  • antiSpoof — FluxVM drops guest traffic whose source IP (and, on a bridged tap, source MAC) is not the Machine's; drops show up as spoof_ip / spoof_mac in kaironctl network drops.
  • learnIP — when Kairon has no guest IP, it uses the address FluxVM learned from the guest's ARP or IPv6 neighbor advertisements, or its DHCP lease (status.network.edge.guestIPSource).
  • qos — each limit is optional; an explicit 0 is rejected. Egress is enforced in the eBPF program, ingress by qdiscs on the host interface. A selecting MachineNetworkPolicy's maxIngress* / maxEgress* fill any limit left unset here.

Setting any of these, or dataplaneMode: ebpf, makes kairon-node post the VM edge to FluxVM each tick. A mode: tap Machine with none of them also gets an edge when a selecting MachineNetworkPolicy sets allowSNI, allowDNS or maxIngress*. FluxVM needs its eBPF dataplane and dataplane schema 12. On a netns Machine, MAC anti-spoof and ARP learning do not apply; use a bridged tap if you need them. Full reference: ebpf-edge.md.

Cilium cluster network (ExternalWorkload)​

spec:
  network:
    mode: tap
    netns: true
    ciliumAttach: true
    dataplaneMode: cilium
    dataplaneRequired: true

Requires Helm network.ciliumAttach.enabled=true. The controller creates a cluster-scoped CiliumExternalWorkload named kairon-<ns>-<name>, projects status.network.cilium, and sets spec.network.podUID from the CEW UID.

spec:
  network:
    mode: macvtap
    parent: eth0
    macvtapMode: bridge   # bridge|vepa|private|passthru

Pod identity (Secure Containers)​

spec:
  network:
    mode: tap
    netns: true
    podUID: "8f3c2e1a-…"   # CRI sandbox UID → FluxVM pod_uid

Cloud-init guest customization​

spec.cloudInit forwards operator-supplied first-boot customization into FluxVM's own cloud-init NoCloud seed image -- SSH keys, hostname, a guest username, packages, and first-boot commands, without needing a custom-baked image:

spec:
  cloudInit:
    hostname: web-01
    user: ops
    sshAuthorizedKeys:
      - "ssh-ed25519 AAAA... ops@laptop"
    packages:
      - nginx
    runCmd:
      - systemctl enable --now nginx
    writeFiles:
      - path: /etc/nginx/conf.d/app.conf
        content: |
          server { listen 8080; location / { proxy_pass http://127.0.0.1:3000; } }
        permissions: "0644"

writeFiles drops a file into the guest before first boot via cloud-init's own write_files module -- e.g. a systemd unit or an app config -- without needing a custom-baked image. permissions is an octal mode string (defaults to cloud-init's own 0644 when unset).

Equivalent flags: kaironctl create ... --hostname web-01 --user ops --ssh-key "ssh-ed25519 AAAA..." --package nginx --runcmd "systemctl enable --now nginx" (--ssh-key/--package/--runcmd are repeatable) -- writeFiles has no kaironctl create flag yet, only the YAML/kubectl path. The dashboard's create form exposes a hostname field and a single SSH key field.

Like forwards above, cloudInit only takes effect when the FluxVM runtime is first created -- it has no effect on an already-running Machine. It requires a cloud-init-aware image (most cloud/server distro images are); FluxVM applies it regardless of image, so a non-cloud-init image simply ignores the seed data.

Service Fabric membership​

Declare VIP backends after the guest IP is known. The named service must already exist in FluxVM; Kairon merges this Machine as a backend.

spec:
  serviceFabric:
    services:
      - name: web-vip
        port: 8080
        weight: 1

Membership is removed again the moment this Machine's guest stops being reachable: deleting the Machine, or setting spec.powerState to Stopped or Halted, all deregister its backend entry from every VIP listed here before the operation completes. This is fail-closed, not best-effort — a Machine stuck unable to reach FluxVM to deregister stays around (deletion) or keeps reporting its last real status (stop/halt) rather than silently finishing while a dead backend, or one a completely different Machine's DHCP-reused guest IP later inherits, is left registered against the VIP.

The same fail-closed pruning also runs on every reconcile tick while the Machine keeps running, not just at delete/stop/halt — so two other real cases are covered too, neither of which needs the Machine to ever stop: removing an entry from serviceFabric.services while the Machine keeps running, and the guest's IP address itself changing (a DHCP re-lease, or a reboot landing on a different lease). Before this pruning existed, an IP change in particular just registered the new address as an additional backend and left the VIP still routing traffic at the old one forever — Kairon now tracks exactly which (service, port, guestIP) triples it has applied in status.appliedServiceFabricMemberships precisely so it can notice and retract one that's no longer current.

Status​

FieldMeaning
status.guestIPGuest address from FluxVM
status.network.guestIP / tapNameSame, nested for Fabric Dataplane tab
status.network.dataplane.attachedTC/eBPF hook attached
status.network.dataplane.modelegacy / ebpf / cilium
status.network.dataplane.identityDataplane identity
status.network.dataplane.schemaVersionBPF schema
status.network.dataplane.policyFingerprintCommitted policy fingerprint
status.network.dataplane.policySyncedMaps match durable policy
status.network.edge.identityStable VM-edge identity (survives IP changes and migration)
status.network.edge.antiSpoof / policyNameEdge anti-spoof requested; MachineNetworkPolicy merged into the edge
status.network.edge.guestIPSourceagent, arp, nd or dhcp
status.network.edge.conntrackRestored / blackholeWindowMsConntrack entries moved by the last live migration, and the gap between export and restore
status.appliedServiceFabricMembershipsGround truth of which (service, port, guestIP) backends Kairon has actually registered -- used to prune stale ones, see above

Node readiness​

Set KAIRON_DATAPLANE_REQUIRED=true on kairon-node so the agent stays NotReady until FluxVM /readyz succeeds (FluxVM itself fail-closes when sandbox.dataplane.required is enabled).

Live migration and network state​

On live migrate, the source agent quiesces and exports network state; the target restores after prepare and resumes after commit. Snapshots travel on the mTLS peer session — not in CRD status. Requires a working migration adapter for memory transfer; see migration-adapter.md.

For Machines with the VM edge, the source also exports FluxVM's live conntrack table onto the migration session and the destination restores it before resume, so established connections keep flowing. A snapshot whose identity does not match the Machine is rejected. See ebpf-edge.md.

Examples​