FluxVM — Product Overview
A host-local VM API — without the platform rewrite. Secure Containers add
Kubernetes RuntimeClass fluxvm: a Pod with its own guest kernel, plus
Sentinel host↔guest policy (sentinel-wedge.md); see the
supported profile.
This doc is the capability tour and the one authoritative metrics table for FluxVM. For the pitch and quick start, see the README; for who this is (and isn't) for, see POSITIONING.md; for the exhaustive feature checklist, see FEATURES.md.
The Problem
Teams that need a host-local VM control plane — CI runners, sandboxed code execution, per-branch environments, Kubernetes VM workloads, or longer-lived guests — need a real API, a real kernel boundary and automatic cleanup, without adopting a whole private-cloud stack.
FluxVM is a control plane with a real API, without requiring a whole private-cloud stack. Disposable / short-lived patterns (TTL, CoW) are available when you want them.
What Is FluxVM?
FluxVM is a Rust-native VM control plane. It provides:
- One binary, one REST API —
fluxvmCLI talks tofluxctl serve - 4 backends behind one trait — QEMU/KVM, Cloud Hypervisor, Firecracker, and the in-tree FluxVM hypervisor, selectable per-VM or via
"backend":"auto" - Optional TTL cleanup —
ttl_secondson a VM spec means a forgotten or crashed job still gets torn down when you opt in - A vsock guest agent —
exec, PTY console, file transfer, with no SSH or network path required
Capability Tour
The exhaustive, line-by-line checklist lives in FEATURES.md. This section is the skim version.
VM lifecycle — create, list, get, pause, resume, delete across all 4 backends; "backend":"auto" resolution; vsock-based exec/console/file-transfer with no SSH required.
Networking — QEMU user-mode NAT, TAP+bridge, per-VM network namespaces, macvtap; Network Fabric is GA (schema v4) — a TC/eBPF or Cilium-coexistence dataplane with IPv4/IPv6 L3+L4 policy, rate limits, security groups, CNP, live reconfigure, and REST observability (native eBPF is the default dataplane; nftables is an explicit mode = "legacy" compatibility mode).
Service Fabric — node-local Maglev VIP load balancing: dual-stack NAT/DSR/SNAT, health-aware routing, incremental reconcile, per-service EDT, flow export.
Images — virt-builder-style build-image (guestkit-based, never libguestfs), per-distro package install, an Ed25519-signed image catalog with REST CRUD, Windows/Kryton golden-image customization with live QGA.
Operations — cgroup v2 resource control (limits, freeze/thaw, PSI), warm VM pools, the Firecracker jailer (chroot + uid/gid drop), alternative storage backends (LVM thin, NBD, Ceph RBD — RBD verified against a real Rook Ceph cluster), admission policy limits, bearer-token auth/RBAC.
Kubernetes & fleet — DisposableVm CRD + node-local operator (verified end to end against a real k3s cluster, 9/9 checks passing); fluxvm-microvm scheduler-native path; fluxvm-agent distributed node-agent with a central fleet registry and load-aware placement, verified across two real, physically separate hosts.
Secure Containers (GA) — containerd runtime-v2 shim (containerd-shim-fluxvm-v2) mapping a Pod/task group onto one FluxVM microVM, with CNI L2, cgroup-v2 stats, VSOCK stdio/TTY, namespaces, device passthrough, guest AppArmor/SELinux/seccomp, and Sentinel NetworkPolicy. Supported profile + flip runbook: secure-containers-supported-profile.md, secure-containers-flip-runtimeclass.md. Not Kata-equivalent — see secure-containers.md.
Agent sandboxes on a Mac — an offline vz sandbox (no network card; commands and files over vsock) ready in about 4.5 s, forked into four running copies in about 4 s, with the four tests run in 0.3 s; with host_gpu the guest reaches the host Metal/MLX OpenAI endpoint over vsock (OPENAI_BASE_URL, host-inference.md). Measured on one Apple M4 mini (macOS 27.2): examples/agent-sandbox-demo; memory per sandbox 205 to 788 MiB resident, plan on about 800 MiB (agent-density.md). Only offline or no-NAT sandboxes fork, and the copies keep the parent's identity. Install: install-macos.md.
Mac cloud platform — tenants and 24-hour Mac leases, quotas, snapshots and clones, persistent volumes, SSH and port tunnels through central, SSO with roles, graphs and Prometheus, warm pools, prices, statements and signed webhooks, Go/Python/TypeScript clients and a Terraform provider (mac-cloud-tenancy.md, mac-cloud-operations.md). Tested against simulated Macs and unit tests; no multi-Mac fleet, real Stripe or identity provider has been run, and self-service sign-up is off by default: it confirms the address by a mailed link through the operator's mail webhook, which has not run against a real mail service.
Sentinel — eBPF host + guest policy/observability for Secure Containers and the Network Fabric edge; buyer wedge: sentinel-wedge.md.
Architecture
+-------------------------+
CLI / REST -------->| Rust VmManager |
| state + TTL + sandboxes |
+------------+------------+
|
+-------------+--------------+--------------+--------------+
| | | | |
+---v---+ +-----v-----+ +-----v------+ +----v-------------+
| QEMU | | Cloud | |Firecracker | | FluxVM hypervisor|
| qcow2 | | Hypervisor| | raw rootfs | | (agent sandboxes)|
+---+---+ +-----+-----+ +-----+------+ +----+-------------+
| | | |
+------+------+--------------+--------------+
|
KVM + TAP/bridge + Linux host
Full diagrams (image pipeline, Secure Containers request flow, Network Fabric packet-decision and control-plane sequence): Architecture at a glance and docs/network-fabric.md.
Deployment Models
Single host, standalone
fluxctl serve on one Linux box with KVM. Suitable for CI runners, dev/test environments, and sandboxed execution on a single machine.
Requirements: Linux, KVM, current stable Rust toolchain to build.
Multi-host fleet (no Kubernetes)
fluxvm-agent provides a central fleet registry plus per-host heartbeat clients, with load-aware placement — verified across two real, physically separate hosts.
Requirements: 2+ hosts, network connectivity between them for the fleet registry.
Kubernetes-native
fluxvm-kube operator reconciling DisposableVm CRs against a local fluxctl serve instance per node, or fluxvm-microvm for scheduler-driven placement via shadow Pods.
Requirements: A Kubernetes cluster (verified against k3s), nodes with /dev/kvm.
Technology Stack
| Layer | Technology |
|---|---|
| Language | Rust |
| VMM backends | QEMU/KVM, Cloud Hypervisor, Firecracker, in-tree fluxvm-hypervisor |
| Networking | netlink (direct), TC/eBPF (Network Fabric), nftables (fallback) |
| Guest agent transport | AF_VSOCK |
| Kubernetes integration | Custom Resource Definitions (DisposableVm, MicroVM), controller-runtime-style operators |
| Image tooling | guestkit-based (no libguestfs dependency) |
Project Statistics
Every figure below is counted directly from source, not estimated.
| Metric | Value | Method |
|---|---|---|
| Crates | 23 | ls crates/ |
| Backends | 4 | QEMU/KVM, Cloud Hypervisor, Firecracker, in-tree FluxVM hypervisor |
| Kubernetes CRD operator validation | 9/9 checks passing | Real k3s cluster; see scripts/test-kube-operator.sh — covers CRD acceptance, live reconciliation to a real running VM, out-of-band-delete self-healing, and finalizer-blocked cleanup with no leaked QEMU process |
| Multi-host fleet validation | Verified across 2 real, physically separate hosts | docs/operations.md |
| Storage backends | 4 | qcow2/raw (default), LVM thin, NBD, Ceph RBD (RBD verified against a real Rook Ceph cluster) |
| Network modes | 4 (+1 opt-in) | user-mode NAT, TAP+bridge, netns+DHCP, macvtap — all 4 SSH-verified end to end in regression tests; bridge-less direct is GA for Secure Containers (FLUXVM_CONTAINER_CNI_DATAPATH=auto|direct|bridge, live Cilium/KVM evidence in direct-datapath.md) |
| Boot latency / VM density / migration downtime | Method published; archived sample numbers below — not a sizing SLA | scripts/record-baseline.sh + benchmarks/evidence/sc-hotcake-bundle-20260926.txt. Firecracker’s 125 ms / 5-per-core figures in capability-figures.md are not FluxVM SLAs. |
Capacity table (archived evidence only)
Every row cites a file under docs/benchmarks/evidence/. Do not treat these as
product SLAs — they are lab archives for planning conversations.
| Metric | Archived value | Evidence |
|---|---|---|
| Warm-pool claim (API, QEMU) | 24 ms to running member | direct-datapath-live-realguest-20260920.txt §5 |
| Bridge vs direct pod↔pod RTT (real guest) | p50 ~0.53 ms vs ~0.46 ms (inside run noise) | same file §3 |
| Bridge vs direct TCP (1 GiB nc) | ~0.34 vs ~0.37 Gbit/s (no clear guest-visible win) | same file §3 |
| flux-vm sandbox cold create (N=4, 256 MiB) | avg ~28.5 s; VMM RSS ~5.5 MiB | density-20260918-80.79.5.173.txt Run B |
| flux-vm sandbox cold create (N=8) | avg ~65 s | same file Run A |
| Policy Observer RSS (idle scrape) | VmRSS ~8.8 MiB | policy-observer-scrape-20260918T035554Z.txt |
| SC k8s userns | Succeeded, distinct_user_ns=1 | sc-live-userns-k8s-20260926.txt |
| SC in-VM distinct userns | distinct_user_ns=2 | sc-live-userns-k8s-multi-20260926.txt |
| SC live finish pass (lab) | pass=7 skip=3 fail=0 | sc-live-phases-summary-20260926.txt |
Bundle index + SHA-256 of members: sc-hotcake-bundle-20260926.txt.
Unlike some of the figures in this table, we deliberately don't include a lines-of-code count or an aggregate REST-endpoint count — neither is currently asserted anywhere in this project's own docs, and inventing one here would just create the kind of stale, unverifiable claim this whole document is trying to avoid.
License
Zyvor Production License v1.0 for the entire repository: non-production use is free; production use requires a commercial license (sales@zyvor.dev). See README.md — License and NOTICE.
For technical details, see docs/api.md, docs/operations.md, and docs/PRODUCTION.md.