FluxVM MicroVM

Schedule a VM like a Pod. Without KubeVirt.

Kubernetes-native MicroVM compute for FluxVM — without KubeVirt. Supports short-lived / disposable jobs when you set TTLs; not limited to them.

The VMM process lives on the host under fluxctl serve. The Pod is only a capacity ticket (registry.k8s.io/pause plus CPU/memory requests). The one privileged surface stays the existing fluxvm-kube DaemonSet that already runs fluxctl serve on capable nodes.


Binary	`fluxvm-microvm`
API group	`microvm.fluxvm.zyvor.io` / `v1alpha1`
Controllers	Namespace `fluxvm-system`
Manifests	`deploy/k8s/microvm/`
Examples	`examples/microvm/`
Tutorials	`docs/tutorials/microvm/`

Vs DisposableVm and KubeVirt

	DisposableVm (`fluxvm-kube`)	MicroVM (`fluxvm-microvm`)	KubeVirt
API group	`fluxvm.zyvor.io`	`microvm.fluxvm.zyvor.io`	`kubevirt.io`
Placement	Explicit `spec.node` (or optional placer)	kube-scheduler via shadow Pod	virt-launcher Pod
Where QEMU/CH/FC runs	Host under `fluxctl serve`	Host under `fluxctl serve`	Inside virt-launcher container
Privileged surface	`fluxvm-kube` DaemonSet	Same DaemonSet + thin node-agent	virt-handler / launcher
Capacity accounting	Operator / placer heuristics	Pod requests (CPU/memory)	Pod + CDI
Jobs / pools	Warm pools via REST	`MicroVMJob`, `MicroVMPool` CRs	Jobs / DataVolumes (different model)
Live migration / CDI / virtctl	No CDI/`virtctl`; QEMU migrate via `fluxctl` / receivers	Same (v1)	Yes

Use DisposableVm when Ragnarok (or another controller) already pins spec.node. Use MicroVM when you want scheduler-driven placement and Job/Pool CRs on the same host VMM. Both talk to local fluxctl serve — neither is KubeVirt.

Architecture

MicroVM CR
    │
    ▼
cluster controller ──► creates shadow Pod (pause + requests)
    │                     kube-scheduler binds Pod → node
    ▼
status.phase=Scheduled, status.runtime.node=<node>
    │
    ▼
node-agent (DaemonSet, hostNetwork)
    │  FLUXVM_URL=http://127.0.0.1:7788
    ▼
fluxctl serve ──► QEMU / Cloud Hypervisor / Firecracker on the host

Phases (typical): Pending → Scheduled → Provisioning → Running (then Succeeded / Failed when persist: false and TTL or command finishes).

Optional: spec.service: true publishes an EndpointSlice once status.guestIP is set.

CRDs

Kind	Short	Purpose
`MicroVM`	`mvm`	Guest. Default `persist: false` — TTL expiry is success.
`MicroVMJob`	`mvmj`	Run-to-completion: child MicroVMs + optional vsock `command`.
`MicroVMPool`	`mvmp`	Node-local warm pool via FluxVM `POST /v1/pools`.
`GuestImage`	`gimg`	Catalog: node agent marks Ready when `spec.source` is a host file; MicroVM `spec.image` may name a GuestImage.

All kinds: apiVersion: microvm.fluxvm.zyvor.io/v1alpha1.

Print or apply CRDs:

fluxvm-microvm --print-crd | kubectl apply -f -
# or
kubectl apply -f deploy/k8s/microvm/crd.yaml

Deploy

1. DaemonSet fluxvm-kube first so fluxctl serve listens on 127.0.0.1:7788 on capable nodes:

kubectl apply -f deploy/k8s/namespace.yaml
kubectl apply -f deploy/k8s/crd.yaml
kubectl apply -f deploy/k8s/rbac.yaml
kubectl apply -f deploy/k8s/configmap.yaml
kubectl label node <node> ragnarok.io/fluxvm-capable=true
kubectl apply -f deploy/k8s/daemonset.yaml

See deploy/k8s/README.md and user/kubernetes-deployment.md.

2. Then MicroVM controllers:

kubectl apply -f deploy/k8s/microvm/
# crd.yaml → rbac.yaml → controller.yaml → node-agent.yaml
# Create Secret fluxvm-microvm-token (key: token) before node-agent when auth is on.

The controller Deployment runs fluxvm-microvm controller without --convert by default (opt-in dual-run bridge). The node-agent DaemonSet selects ragnarok.io/fluxvm-capable=true and uses NODE_NAME + FLUXVM_URL.

Shadow Pods request a tiny pause budget (10m CPU / 32Mi RAM), not the guest’s vCPU/memory — guest resources live under the host VMM cgroup.

Local / lab without the full DaemonSet image:

fluxvm-microvm --print-crd | kubectl apply -f -
fluxvm-microvm controller &
NODE_NAME=$(hostname) FLUXVM_URL=http://127.0.0.1:7788 fluxvm-microvm node-agent

First MicroVM

Stage an image on the node (same path you would use for DisposableVm), then:

kubectl apply -f examples/microvm/microvm.yaml
kubectl get mvm sandbox-42 -w

Example (examples/microvm/microvm.yaml):

apiVersion: microvm.fluxvm.zyvor.io/v1alpha1
kind: MicroVM
metadata:
  name: sandbox-42
spec:
  backend: qemu
  image: /var/lib/fluxvm/images/ubuntu.qcow2
  vcpus: 2
  memoryMib: 2048
  networkMode: tap
  bridge: vmbr0
  netns: true
  service: true
  servicePort: 22
  ttlSeconds: 900
  persist: false

Adjust image, bridge, and backend to match the node. Prefer networkMode: user or none for a first smoke test if TAP/bridge is not ready.

For a bridge-less tap on a physical uplink (no vmbr0), use networkMode: direct: parent names the unenslaved uplink NIC and guestIps lets the LAN discover the guest by ARP (needs sandbox.dataplane.mode = ebpf; see direct-datapath.md for the bounds):

  networkMode: direct
  parent: enp1s0
  mac: "02:00:00:00:0a:0a"
  guestIps: ["192.168.1.50"]

MicroVMJob

Run-to-completion: creates child MicroVMs from spec.template, tracks completions / parallelism / backoffLimit. Optional template.command runs over vsock once Running; with persist: false the guest is deleted after the command.

kubectl apply -f examples/microvm/microvmjob.yaml
kubectl get mvmj compile-main -w

Set spec.ttlSecondsAfterFinished to have the Job controller delete a finished MicroVMJob (and, via ownerReferences, its child MicroVMs) on its own once it has sat in Succeeded/Failed for that long — the same ttlSecondsAfterFinished pattern as a stock Kubernetes Job. Without it, a finished MicroVMJob (and its child MicroVMs) stays around until something deletes it by hand, which is exactly what CI-style fan-out workloads (examples/microvm/microvmjob.yaml's own use case) tend to create a lot of. The clock starts on the first reconcile that observes a terminal phase (status.finishedAt, set once and never reset) rather than on every reconcile, so restarting the controller does not restart the countdown.

Tutorial: tutorials/microvm/02-job.md.

MicroVMPool

Maps to FluxVM warm pools (POST /v1/pools). Claim with spec.claimFrom: <pool-name> on a MicroVM (or Job template).

kubectl apply -f examples/microvm/pool.yaml
kubectl get mvmp ci-warm -w

Tutorial: tutorials/microvm/03-pool.md.

GuestImage

GuestImage catalogs a disk (spec.source, optional sha256 / backend / kernel). No CDI / importer Pod — GuestKit (or ops) stages the file on the node. The node-agent reconciler sets status.ready + status.path when spec.source is an absolute path that exists on that host.

When spec.sha256 is set, the staged file's digest is checked before status.ready flips true — a wrong or corrupted file under the right path never becomes Ready, so it can't be handed to every MicroVM naming this catalog entry. A mismatch leaves status.ready=false with status.message: "sha256 mismatch: expected <spec>, got <actual>"; status never carries a path until the digest checks out. The digest is only recomputed when the file's size/mtime changes (cached in status.verifiedSignature), so a large disk image is hashed once, not on every 30s reconcile.

When spec.kernel is set — a host path to a vmlinux-style kernel for Firecracker's direct-kernel boot — its presence on the node is confirmed the same way before status.ready flips true, and the confirmed path is recorded in status.kernelPath. Every MicroVM that resolves its spec.image to this catalog entry gets status.kernelPath forwarded as kernel on the underlying fluxctl serve create request automatically — there's nothing to set on the MicroVM itself. A GuestImage that names a kernel but doesn't have one staged is never marked Ready (fail closed): a missing kernel must block the whole catalog entry, not launch a Firecracker guest with no kernel or a stale one left over from a previous entry with the same name. spec.kernel is unused by backends that boot straight off the disk image (QEMU, Cloud Hypervisor with firmware) — leave it unset for those.

MicroVM.spec.image may be:

a direct path / URL / *.qcow2|raw|ext4|img name (used as-is), or
a GuestImage name in the same namespace (resolved to status.path once Ready)

kubectl apply -f - <<'EOF'
apiVersion: microvm.fluxvm.zyvor.io/v1alpha1
kind: GuestImage
metadata: {name: ubuntu-lab}
spec: {source: /var/lib/fluxvm/images/ubuntu.qcow2}
EOF
# MicroVM.spec.image: ubuntu-lab

DisposableVm bridge (`--convert`, opt-in)

fluxvm-microvm controller --convert

Watches DisposableVm (fluxvm.zyvor.io) and creates a same-name MicroVM with persist: true and nodeName copied. Converted guests get:

microvm.fluxvm.zyvor.io/converted-from=disposablevm
microvm.fluxvm.zyvor.io/driven-by=fluxvm-kube — the MicroVM node agent skips POST /v1/vms so fluxvm-kube remains the sole VMM driver

Deploy manifests leave --convert off. Enable only when you understand the driven-by skip. Tutorial: tutorials/microvm/04-convert-disposablevm.md.

Verify

kubectl -n fluxvm-system get pods -o wide
kubectl -n fluxvm-system logs deploy/fluxvm-microvm-controller --follow
kubectl -n fluxvm-system logs ds/fluxvm-microvm-node --follow
kubectl get crd | grep microvm.fluxvm.zyvor.io
kubectl get mvm,mvmj,mvmp -A
curl -sf http://127.0.0.1:7788/readyz | jq .
# Schedule→Running histograms (default MICROVM_METRICS_ADDR=127.0.0.1:9108):
curl -sf http://127.0.0.1:9108/metrics | head

Tests:

cargo test -p fluxvm-microvm
./scripts/test-microvm.sh
python3 scripts/test-microvm-policy.py
./scripts/test-microvm-k8s-smoke.sh   # lab k3s

Limitations (v1)

No virt-launcher, CDI, or virtctl. QEMU live migration uses fluxctl migrate plus POST /v1/migration/receivers (not KubeVirt migration parity).
No second privileged VMM DaemonSet — reuse fluxvm-kube / host fluxctl serve.
No k8s-native image pull — host-staged paths only (GuestImage Ready when the file exists on the node; HTTP sources are staged on the node and are not CDI DataVolumes).
kubectl-fluxvm resolves spec.node / runtime node for console/exec/pause/resume; node names are validated as SSH hostnames (ssh -- host -- …) to block option injection.
Shadow Pod is capacity accounting, not the VMM process.
Images and TAP/bridges must exist on the scheduled node before Running.

Tutorials

Tutorial	Focus
01 — Getting started	print-crd, deploy, create MicroVM, watch phase
02 — MicroVMJob	Run-to-completion jobs
03 — MicroVMPool	Warm pools + claimFrom
04 — Convert DisposableVm	`--convert` bridge (opt-in)
05 — GuestImage	Host-file Ready + catalog name

Also: kubernetes-deployment.md · deploy/k8s/microvm/README.md · benchmarks/README.md · PRODUCTION.md.

Vs DisposableVm and KubeVirt​

Architecture​

CRDs​

Deploy​

First MicroVM​

MicroVMJob​

MicroVMPool​

GuestImage​

DisposableVm bridge (--convert, opt-in)​

Verify​

Limitations (v1)​

Tutorials​