Skip to main content

FluxVM — Product Overview

A host-local VM API — without the platform rewrite. Secure Containers add Kubernetes RuntimeClass fluxvm: a Pod with its own guest kernel, plus Sentinel host↔guest policy (sentinel-wedge.md); see the supported profile.

This doc is the capability tour and the one authoritative metrics table for FluxVM. For the pitch and quick start, see the README; for who this is (and isn't) for, see POSITIONING.md; for the exhaustive feature checklist, see FEATURES.md.

The Problem​

Teams that need a host-local VM control plane — CI runners, sandboxed code execution, per-branch environments, Kubernetes VM workloads, or longer-lived guests — need a real API, a real kernel boundary and automatic cleanup, without adopting a whole private-cloud stack.

FluxVM is a control plane with a real API, without requiring a whole private-cloud stack. Disposable / short-lived patterns (TTL, CoW) are available when you want them.


What Is FluxVM?​

FluxVM is a Rust-native VM control plane. It provides:

  • One binary, one REST API — fluxvm CLI talks to fluxctl serve
  • 4 backends behind one trait — QEMU/KVM, Cloud Hypervisor, Firecracker, and the in-tree FluxVM hypervisor, selectable per-VM or via "backend":"auto"
  • Optional TTL cleanup — ttl_seconds on a VM spec means a forgotten or crashed job still gets torn down when you opt in
  • A vsock guest agent — exec, PTY console, file transfer, with no SSH or network path required

Capability Tour​

The exhaustive, line-by-line checklist lives in FEATURES.md. This section is the skim version.

VM lifecycle — create, list, get, pause, resume, delete across all 4 backends; "backend":"auto" resolution; vsock-based exec/console/file-transfer with no SSH required.

Networking — QEMU user-mode NAT, TAP+bridge, per-VM network namespaces, macvtap; Network Fabric is GA (schema v4) — a TC/eBPF or Cilium-coexistence dataplane with IPv4/IPv6 L3+L4 policy, rate limits, security groups, CNP, live reconfigure, and REST observability (native eBPF is the default dataplane; nftables is an explicit mode = "legacy" compatibility mode).

Service Fabric — node-local Maglev VIP load balancing: dual-stack NAT/DSR/SNAT, health-aware routing, incremental reconcile, per-service EDT, flow export.

Images — virt-builder-style build-image (guestkit-based, never libguestfs), per-distro package install, an Ed25519-signed image catalog with REST CRUD, Windows/Kryton golden-image customization with live QGA.

Operations — cgroup v2 resource control (limits, freeze/thaw, PSI), warm VM pools, the Firecracker jailer (chroot + uid/gid drop), alternative storage backends (LVM thin, NBD, Ceph RBD — RBD verified against a real Rook Ceph cluster), admission policy limits, bearer-token auth/RBAC.

Kubernetes & fleet — DisposableVm CRD + node-local operator (verified end to end against a real k3s cluster, 9/9 checks passing); fluxvm-microvm scheduler-native path; fluxvm-agent distributed node-agent with a central fleet registry and load-aware placement, verified across two real, physically separate hosts.

Secure Containers (GA) — containerd runtime-v2 shim (containerd-shim-fluxvm-v2) mapping a Pod/task group onto one FluxVM microVM, with CNI L2, cgroup-v2 stats, VSOCK stdio/TTY, namespaces, device passthrough, guest AppArmor/SELinux/seccomp, and Sentinel NetworkPolicy. Supported profile + flip runbook: secure-containers-supported-profile.md, secure-containers-flip-runtimeclass.md. Not Kata-equivalent — see secure-containers.md.

Agent sandboxes on a Mac — an offline vz sandbox (no network card; commands and files over vsock) ready in about 4.5 s, forked into four running copies in about 4 s, with the four tests run in 0.3 s; with host_gpu the guest reaches the host Metal/MLX OpenAI endpoint over vsock (OPENAI_BASE_URL, host-inference.md). Measured on one Apple M4 mini (macOS 27.2): examples/agent-sandbox-demo; memory per sandbox 205 to 788 MiB resident, plan on about 800 MiB (agent-density.md). Only offline or no-NAT sandboxes fork, and the copies keep the parent's identity. Install: install-macos.md.

Mac cloud platform — tenants and 24-hour Mac leases, quotas, snapshots and clones, persistent volumes, SSH and port tunnels through central, SSO with roles, graphs and Prometheus, warm pools, prices, statements and signed webhooks, Go/Python/TypeScript clients and a Terraform provider (mac-cloud-tenancy.md, mac-cloud-operations.md). Tested against simulated Macs and unit tests; no multi-Mac fleet, real Stripe or identity provider has been run, and self-service sign-up is off by default: it confirms the address by a mailed link through the operator's mail webhook, which has not run against a real mail service.

Sentinel — eBPF host + guest policy/observability for Secure Containers and the Network Fabric edge; buyer wedge: sentinel-wedge.md.


Architecture​

+-------------------------+
CLI / REST -------->| Rust VmManager |
| state + TTL + sandboxes |
+------------+------------+
|
+-------------+--------------+--------------+--------------+
| | | | |
+---v---+ +-----v-----+ +-----v------+ +----v-------------+
| QEMU | | Cloud | |Firecracker | | FluxVM hypervisor|
| qcow2 | | Hypervisor| | raw rootfs | | (agent sandboxes)|
+---+---+ +-----+-----+ +-----+------+ +----+-------------+
| | | |
+------+------+--------------+--------------+
|
KVM + TAP/bridge + Linux host

Full diagrams (image pipeline, Secure Containers request flow, Network Fabric packet-decision and control-plane sequence): Architecture at a glance and docs/network-fabric.md.


Deployment Models​

Single host, standalone​

fluxctl serve on one Linux box with KVM. Suitable for CI runners, dev/test environments, and sandboxed execution on a single machine.

Requirements: Linux, KVM, current stable Rust toolchain to build.

Multi-host fleet (no Kubernetes)​

fluxvm-agent provides a central fleet registry plus per-host heartbeat clients, with load-aware placement — verified across two real, physically separate hosts.

Requirements: 2+ hosts, network connectivity between them for the fleet registry.

Kubernetes-native​

fluxvm-kube operator reconciling DisposableVm CRs against a local fluxctl serve instance per node, or fluxvm-microvm for scheduler-driven placement via shadow Pods.

Requirements: A Kubernetes cluster (verified against k3s), nodes with /dev/kvm.


Technology Stack​

LayerTechnology
LanguageRust
VMM backendsQEMU/KVM, Cloud Hypervisor, Firecracker, in-tree fluxvm-hypervisor
Networkingnetlink (direct), TC/eBPF (Network Fabric), nftables (fallback)
Guest agent transportAF_VSOCK
Kubernetes integrationCustom Resource Definitions (DisposableVm, MicroVM), controller-runtime-style operators
Image toolingguestkit-based (no libguestfs dependency)

Project Statistics​

Every figure below is counted directly from source, not estimated.

MetricValueMethod
Crates23ls crates/
Backends4QEMU/KVM, Cloud Hypervisor, Firecracker, in-tree FluxVM hypervisor
Kubernetes CRD operator validation9/9 checks passingReal k3s cluster; see scripts/test-kube-operator.sh — covers CRD acceptance, live reconciliation to a real running VM, out-of-band-delete self-healing, and finalizer-blocked cleanup with no leaked QEMU process
Multi-host fleet validationVerified across 2 real, physically separate hostsdocs/operations.md
Storage backends4qcow2/raw (default), LVM thin, NBD, Ceph RBD (RBD verified against a real Rook Ceph cluster)
Network modes4 (+1 opt-in)user-mode NAT, TAP+bridge, netns+DHCP, macvtap — all 4 SSH-verified end to end in regression tests; bridge-less direct is GA for Secure Containers (FLUXVM_CONTAINER_CNI_DATAPATH=auto|direct|bridge, live Cilium/KVM evidence in direct-datapath.md)
Boot latency / VM density / migration downtimeMethod published; archived sample numbers below — not a sizing SLAscripts/record-baseline.sh + benchmarks/evidence/sc-hotcake-bundle-20260926.txt. Firecracker’s 125 ms / 5-per-core figures in capability-figures.md are not FluxVM SLAs.

Capacity table (archived evidence only)​

Every row cites a file under docs/benchmarks/evidence/. Do not treat these as product SLAs — they are lab archives for planning conversations.

MetricArchived valueEvidence
Warm-pool claim (API, QEMU)24 ms to running memberdirect-datapath-live-realguest-20260920.txt §5
Bridge vs direct pod↔pod RTT (real guest)p50 ~0.53 ms vs ~0.46 ms (inside run noise)same file §3
Bridge vs direct TCP (1 GiB nc)~0.34 vs ~0.37 Gbit/s (no clear guest-visible win)same file §3
flux-vm sandbox cold create (N=4, 256 MiB)avg ~28.5 s; VMM RSS ~5.5 MiBdensity-20260918-80.79.5.173.txt Run B
flux-vm sandbox cold create (N=8)avg ~65 ssame file Run A
Policy Observer RSS (idle scrape)VmRSS ~8.8 MiBpolicy-observer-scrape-20260918T035554Z.txt
SC k8s usernsSucceeded, distinct_user_ns=1sc-live-userns-k8s-20260926.txt
SC in-VM distinct usernsdistinct_user_ns=2sc-live-userns-k8s-multi-20260926.txt
SC live finish pass (lab)pass=7 skip=3 fail=0sc-live-phases-summary-20260926.txt

Bundle index + SHA-256 of members: sc-hotcake-bundle-20260926.txt.

Unlike some of the figures in this table, we deliberately don't include a lines-of-code count or an aggregate REST-endpoint count — neither is currently asserted anywhere in this project's own docs, and inventing one here would just create the kind of stale, unverifiable claim this whole document is trying to avoid.


License​

Zyvor Production License v1.0 for the entire repository: non-production use is free; production use requires a commercial license (sales@zyvor.dev). See README.md — License and NOTICE.


For technical details, see docs/api.md, docs/operations.md, and docs/PRODUCTION.md.