Skip to main content

REST API reference

Start the server:

sudo /usr/local/bin/fluxctl --config /etc/fluxvm.toml serve

Default bind address:

127.0.0.1:7788

Endpoints​

GET /healthz
GET /readyz
GET /metrics
GET /v1/openapi.json # OpenAPI 3.1, VM surface (no auth)
POST /v1/vms # ?ready=exec adds first_command_ms + phases; Idempotency-Key honored
GET /v1/vms
GET /v1/vms?name=<name>
GET /v1/vms?tenant=<tenant>
GET /v1/vms?label=<selector> # env=prod,team!=x,gpu,!tmp
GET /v1/vms/{uuid}
PATCH /v1/vms/{uuid} # {"name": "...", "labels": {"k": "v", "gone": null}}
POST /v1/vms/{uuid}/start
POST /v1/vms/{uuid}/restart
POST /v1/vms/{uuid}/clone # {"name": "..."}; source must be stopped
POST /v1/vms/{uuid}/fork # {"count": 1-32, "namePrefix": "..."}; admin; running flux-vm source; 201 {"items": [...], "elapsed_ms": n}; ?ready=exec; Idempotency-Key honored
POST /v1/vms/{uuid}/backup # {"name": "...", "compress": bool, "all_disks": bool, "quiesce": "auto"|"required"|"never"}
POST /v1/vms/{uuid}/restore-backup # {"name": "..."}; VM must be stopped
GET /v1/backups # newest first
DELETE /v1/backups/{name}
GET /v1/backups/{name}/root # the backup's standalone root qcow2 (download)
POST /v1/vms/{uuid}/start-from-snapshot
POST /v1/vms/{uuid}/snapshot # Idempotency-Key honored
POST /v1/vms/{uuid}/restore # {"tag": "..."}; admin; running flux-vm restores in place (memory + disk)
GET /v1/vms/{uuid}/snapshots
DELETE /v1/vms/{uuid}/snapshots/{tag}
GET /v1/vms/{uuid}/disks
POST /v1/vms/{uuid}/disks # {"name": "data", "size_gib": 10}, {"name": "pvc", "path": "/dev/rbd0"} or {"name": "disk1", "backing": "/var/lib/fluxvm/images/imported/web01/disk1.raw"} (qcow2 overlay); optional "io" (QEMU disk I/O, below)
PATCH /v1/vms/{uuid}/disks/{name} # {"size_gib": 20}; name "root" = boot disk
DELETE /v1/vms/{uuid}/disks/{name} # deletes the file (or overlay); only unlinks a path-attached disk
PUT /v1/vms/{uuid}/disks/{name}/throttle # {"throttle": {"iops_total": 500}}; null lifts the limits (QEMU)
# vz: also "kind" (image|block|nbd), "url", "read_only", "caching", "sync", "controller"
# (virtio|nvme|usb); applies at the next start (a USB image disk at once); see macos.md#extra-disks
POST /v1/vms/{uuid}/cdroms/{name}/eject # remove install media (live when running); the empty drive stays
GET /v1/vms/{uuid}/serial # websocket, raw serial bytes (QEMU)
# GET /v1/vms leaves out container warm-pool slots unless ?all=true or label=fluxvm.oci-warm...
GET /v1/vms/{uuid}/ports/{name} # websocket, raw bytes to and from a vz console port (apple.console_ports; admin)
POST /v1/vms/{uuid}/stop
POST /v1/vms/{uuid}/pause
POST /v1/vms/{uuid}/resume
POST /v1/vms/{uuid}/hibernate # qemu; save RAM/CPU/devices, stop QEMU; next start resumes; admin; see hibernation.md
DELETE /v1/vms/{uuid}/hibernation # drop the saved state; next start boots from disk; admin
POST /v1/vms/import-hibernated # {"id"}; adopt a VM hibernated on another node from the shared dir; admin
GET /v1/stats # one sample of this host and every VM (cumulative counters) for the fleet's metrics collector; see "Node stats"
GET /v1/volumes # persistent volumes on this host; admin; see "Volumes and tunnels (fleet platform)"
POST /v1/volumes # {"name": "data", "size_gib": 50}; sparse raw file; 201; 409 when the name is taken; admin
GET /v1/volumes/{name} # admin
DELETE /v1/volumes/{name} # 409 while any VM on this host references it; admin
POST /v1/volumes/{name}/resize # {"size_gib": 100}; grow only, and only while no VM references it; admin
GET /v1/vms/{uuid}/tunnel?port=N # websocket: one TCP connection to port N of a running VM (binary messages); admin; see "Volumes and tunnels (fleet platform)"
GET /v1/host/hibernation # {facts: {qemu_version, machine_type, cpu_vendor, cpu_flags, state_dir}}
POST /v1/host/hibernation/check # {"manifest"} -> {compatible, reasons}
POST /v1/vms/{uuid}/diagnostics # {pcap_seconds, pcap_filter, memory_dump, redact, log_lines} -> 202 bundle job; 409 if one is running; admin; see diagnostics-bundle.md
GET /v1/vms/{uuid}/diagnostics # bundles kept for the VM (last 5); admin
GET /v1/vms/{uuid}/diagnostics/{token} # job: state running|done|failed, sections, sha256, memory_dump; admin
DELETE /v1/vms/{uuid}/diagnostics/{token} # remove a finished bundle and its dump; 409 while running; admin
GET /v1/vms/{uuid}/diagnostics/{token}/bundle # tar.gz; 409 while running or failed; admin
GET /v1/vms/{uuid}/diagnostics/{token}/memory-dump # ELF guest memory dump; admin
POST /v1/vms/{uuid}/resources
POST /v1/vms/{uuid}/hotplug/cpu
POST /v1/vms/{uuid}/hotplug/memory
POST /v1/vms/{uuid}/hotplug/nic
POST /v1/vms/{uuid}/hotplug/nic/unplug # {"mac": ...} or {"tap": ...}; extra NICs only
POST /v1/vms/{uuid}/hotplug/share
GET /v1/vms/{uuid}/cpuset
GET /v1/vms/{uuid}/numa # qemu; guest NUMA nodes, live vCPU thread affinity, memdev binding; see host-topology.md
POST /v1/vms/{uuid}/freeze
POST /v1/vms/{uuid}/thaw
GET /v1/vms/{uuid}/frozen
GET /v1/vms/{uuid}/stats
GET /v1/vms/{uuid}/pressure
GET /v1/vms/{uuid}/balloon # flux-vm KVM engine, vz, qemu with balloon; {memory_mib, target_mib, actual_mib}
POST /v1/vms/{uuid}/balloon # {"balloon_mib": N}; admin; 0 deflates
GET /v1/vms/{uuid}/memory # VMM-process PSS/private/shared + balloon; see memory-density.md
GET /v1/vms/{uuid}/logs
GET /v1/vms/{uuid}/console
POST /v1/vms/{uuid}/agent # {"command", "timeout_seconds"?, "policy"?}; see guest-exec-policy.md
POST /v1/vms/{uuid}/agent/ping
POST /v1/vms/{uuid}/agent/put-file
POST /v1/vms/{uuid}/agent/get-file
GET /v1/vms/{uuid}/screenshot # vz; ?max_width=N; image/png of the guest display; admin
POST /v1/vms/{uuid}/input # vz; {"action": "type"|"key"|"click"|...} or {"actions": [...]}; admin
PUT /v1/vms/{uuid}/signin # vz; {"username"?, "password"} into the host login Keychain; admin
GET /v1/vms/{uuid}/signin # {"configured", "username"?}; never the password; admin
DELETE /v1/vms/{uuid}/signin # {"deleted": bool}; admin
POST /v1/vms/{uuid}/signin # {"mode"?: "password"|"username"|"username_tab", "submit"?: bool}; types it; admin
GET /v1/vms/{uuid}/qga/network-interfaces
DELETE /v1/vms/{uuid} # Idempotency-Key honored
GET /v1/vm-templates
POST /v1/vm-templates # {"name", "description"?, "spec", "replace"?}
GET /v1/vm-templates/{name}
DELETE /v1/vm-templates/{name}
POST /v1/vm-templates/{name}/instantiate # {"name": "vm-1", "labels"?: {...}}
GET /v1/events # ?vm=&event=<prefix>&since=<rfc3339>&limit=
GET /v1/events/stream # Server-Sent Events, same filters
GET /v1/quotas/me
POST /v1/images/build
POST /v1/images/import # {"source": "/path/vm.ova", "name": "...", "repair": true, "remove_vmware_tools": false}; admin; 201; see import-vmware.md
GET /v1/images/catalog
POST /v1/images/catalog
DELETE /v1/images/catalog/{name}
POST /v1/images/catalog/{name}/rename
POST /v1/images/catalog/{name}/clone
POST /v1/images/catalog/{name}/export
POST /v1/images/catalog/{name}/read-only
POST /v1/images/catalog/clean
GET /v1/oci/images # cached OCI rootfs images for vz sandboxes; see oci-sandboxes.md
POST /v1/oci/images # {"image": "alpine:3.22", "platform"?: "linux/amd64"}; admin; pull and build the rootfs
DELETE /v1/oci/images/{digest|reference} # admin
POST /v1/oci/prune # admin; drop images no sandbox was started from, and unused blobs
GET /v1/vznets # private vz networks (apple.networks): subnet, members, switch_running; see macos.md
POST /v1/pools
GET /v1/pools
GET /v1/pools/{name}
DELETE /v1/pools/{name}
POST /v1/pools/{name}/claim # ?ready=exec adds first_command_ms + phases
POST /v1/pools/{name}/resize
POST /v1/sandboxes # optional volumes; vz: "offline", "host_gpu" (host-inference.md), "oci": {...} (oci-sandboxes.md)
GET /v1/sandboxes
POST /v1/sandboxes/{id}/snapshot
POST /v1/sandboxes/{id}/fs/read
POST /v1/sandboxes/{id}/fs/write
POST /v1/sandboxes/{id}/process # {"command", "timeout_seconds"?, "policy"?} or {"process": {"argv", "env"?, "cwd"?}} (no shell; agent); see guest-exec-policy.md
GET /v1/sandboxes/{id}/logs # ?lines=N (200); console tail + container exit_code / init_error / restarts / health
POST /v1/sandboxes/{id}/dry-run # {"command", "paths"}; procbox: workspace copy; flux-vm/qemu/firecracker: snapshot/restore
POST /v1/sandboxes/{id}/speculate # {"command", "timeout_seconds"?, "paths"?, "ttl_seconds"?}; admin; returns a pending changeset
GET /v1/sandboxes/{id}/changesets # {"items": [...]}, newest first
GET /v1/sandboxes/{id}/changesets/{cs}
POST /v1/sandboxes/{id}/changesets/{cs}/approve
POST /v1/sandboxes/{id}/changesets/{cs}/reject
POST /v1/sandboxes/{id}/changesets/{cs}/apply # approved only; 409 on conflict
POST /v1/sandboxes/{id}/grants # credential broker grant; admin; 201 GrantInfo
GET /v1/sandboxes/{id}/grants # {"grants": [...]}, no secrets
DELETE /v1/sandboxes/{id}/grants # revoke all; {"revoked": bool}
DELETE /v1/sandboxes/{id}/grants/{grant_id} # revoke one; 404 if unknown
ANY /v1/sandboxes/{id}/http/{port}/{*path}
ANY /sandbox/{id}/{*path}
GET /v1/vms/{uuid}/network/policy
POST /v1/vms/{uuid}/network/policy
GET /v1/vms/{uuid}/network/effective
GET /v1/vms/{uuid}/network/egress # QEMU-process egress policy (user-mode guests): mode + rules, source, what the kernel enforces
PUT /v1/vms/{uuid}/network/egress # {"mode":"loopback|allowlist|audit|open","rules":[{"cidr","proto","port","note"}]} (admin)
DELETE /v1/vms/{uuid}/network/egress # back to the node default (admin)
GET /v1/vms/{uuid}/network/egress/observed # destinations refused (or, in audit mode, that would be)
POST /v1/vms/{uuid}/network/egress/suggest # rules that would allow what was observed; ?apply=true adds them (admin)
GET /v1/network/egress/defaults # node default policy
PUT /v1/network/egress/defaults # ?apply_running=true re-programs running VMs without their own policy (admin)
GET /v1/vms/{uuid}/network/status
GET /v1/vms/{uuid}/network/stats
GET /v1/vms/{uuid}/network/flows
POST /v1/vms/{uuid}/network/edge # Kairon VM-edge spec (admin); enforced and persisted, see vm-edge-contract.md
GET /v1/vms/{uuid}/network/conntrack # live conntrack export (admin); empty entries when not attached
POST /v1/vms/{uuid}/network/conntrack # restore (admin); identity mismatch is rejected, applied on attach if not attached
GET /v1/vms/{uuid}/network/learned-ip # {"ip","source"}: arp|nd from the datapath, else dhcp|fluxvm
GET /v1/vms/{uuid}/network/drops?limit=N # attributed drops with Kairon reason names (default 100, max 4096)
POST /v1/vms/{uuid}/network/capture # starts a 1-30s tcpdump capture on the VM edge (admin)
GET /v1/vms/{uuid}/network/capture # capture sessions and their state (admin)
GET /v1/vms/{uuid}/network/capture/{token} # the finished pcap; 409 while running (admin)
GET /v1/network/groups
POST /v1/network/groups
GET /v1/network/groups/{name}
DELETE /v1/network/groups/{name}
GET /v1/network/cnp
POST /v1/network/cnp
GET /v1/network/cnp/{name}
DELETE /v1/network/cnp/{name}
GET /v1/network/identities
GET /v1/network/observe
GET /v1/network/health
GET /v1/network/ipcache
GET /v1/network/ipam
GET /v1/network/vnets # virtual networks applied on this host; see virtual-network.md
GET /v1/network/vnets/{vni}
PUT /v1/network/vnets/{vni} # admin; replaces the config for the VNI
DELETE /v1/network/vnets/{vni} # admin; refused while a VM here is attached
POST /v1/network/refresh-dns
GET /console/ # FluxVM Studio web UI (static, no auth; it asks for a token); /console redirects here; see ui.md
POST /v1/vms/{id}/ws-ticket # {"kind":"console"|"serial"|"vnc"} -> single-use 30 s ticket for a browser WebSocket (vnc: QEMU machines only)
GET /console/ws/{ticket} # redeem a ticket (WebSocket upgrade; no Authorization needed)

Sandbox routes are the agent-sandbox surface on the FluxVm backend — see agent-sandbox-gaps.md and the Feature highlights.

GET /v1/vms?name=<name> exact-matches on VmRecord.name server-side. POST /v1/vms/{uuid}/start relaunches a Stopped VM from its existing disk/seed, skipping the image-clone/cloud-init/token-inject work create does — for a name-keyed register-then-start caller that already has a VM on disk it just needs running again.

GET /metrics returns Prometheus text-exposition-format gauges: fluxvm_vms_total{status="..."}, fluxvm_vms_by_backend{backend="..."}, and fluxvm_vms_agent_enabled — point a Prometheus scrape_config at it directly, no exporter needed.

GET /v1/vms/{uuid}/logs?lines=N&follow=true streams the VM's captured console output (raw serial, no per-line structure) as chunked plain text — lines (default 100) controls how much history to send before either ending (default) or switching to a live tail (follow=true, polling the log file every 300ms for new lines). Verified against a real booting VM: both the initial tail and the live follow stream return real, growing boot output.

Day-2 VM routes​

Semantics, limits and CLI equivalents for restart, PATCH/labels, snapshots, clone, disks, serial, backup, scheduled snapshots, VM templates and events are in operations.md → Day-2 VM operations. Highlights:

  • GET /v1/vms/{uuid}/serial is a websocket to the QEMU serial port: binary and text frames are written to the guest verbatim, guest output comes back as binary frames. One client at a time; no guest agent needed.
  • POST /v1/vms/{uuid}/backup always writes under state_dir/backups/ (the API never takes a destination path) and returns {name, path, size_bytes, live, quiesced, disks[]}. An optional name replaces the default <vm>-<utc> so callers can find their backup again after a timeout. all_disks skips linked images and block devices, which FluxVM doesn't own. quiesce (default auto) freezes the guest's filesystems through QGA around the snapshot when the agent answers; required fails without it. GET /v1/backups lists backups with that metadata; POST /v1/vms/{uuid}/restore-backup copies one back into a stopped VM, in the disk's own format. Any engine, on default storage or a shared disk file; a running VM can be backed up only on QEMU with default storage.
  • POST /v1/vm-templates/{name}/instantiate goes through the same tenant, created_by_token and token-quota handling as POST /v1/vms. VM templates are separate from the sandbox /v1/templates routes.
  • POST /v1/vms/{uuid}/fork snapshots a running flux-vm VM once and starts count children from that snapshot. Children share the read-only memory file and get a copy-on-write copy of the disk, a new CID, vsock socket and tap. They keep the parent's MAC and guest IP, so the source must use network = none, user, or tap with a per-VM netns, and no extra NICs. Either all children start or none are left behind. Token quotas are charged per child. scripts/bench-fork.sh times it.
  • GET /v1/vms/{uuid}/screenshot and POST /v1/vms/{uuid}/input let an agent drive the display of a running vz VM (Linux or macOS guest) without a console window or host Screen Recording/Accessibility permissions. The PNG's size is in x-fluxvm-screen-width/-height; x-fluxvm-screen-blank: true means the guest has drawn nothing yet. Input actions: type (text, US layout, printable ASCII plus newline and tab, up to 4096 characters), key (key: a character or enter, tab, escape, backspace, delete, arrows, home, end, page_up, page_down, f1-f12; modifiers: shift, control, option, command), move, click (with modifiers), double_click, right_click, middle_click, down, up, drag (to_x, to_y) and scroll (dx, dy wheel notches, optional x, y). Coordinates are screenshot pixels from the top left; after ?max_width=N pass "screen_width": N so they are scaled back. A batch is validated as a whole before anything is sent (at most 100 actions).
  • /v1/vms/{uuid}/signin keeps a guest sign-in for a vz VM in the host's login Keychain (service dev.zyvor.fluxvm.vm-signin, account: the VM id) and types it into the display on POST, so an agent can get past a login screen without ever seeing the password. mode is password (default; the focused field), username (username, Enter, password: a text console) or username_tab (username, Tab, password: a graphical form); submit (default true) presses Enter afterwards. Username and password are printable ASCII, at most 256 characters. Deleting the VM removes the item.
  • GET /v1/events/stream emits event: <name> / data: <json> frames; tenant-scoped tokens only see their tenant's VMs.
  • GET /v1/openapi.json is exempt from auth, like /healthz and /readyz.

Auth / RBAC​

[[auth.tokens]] entries in the config (see config.example.toml) enable bearer-token auth on every route except GET /healthz, GET /readyz (liveness vs readiness), GET /v1/openapi.json, the Studio pages under GET /console/, and the two routes whose path carries their own credential: /preview/{token}/... (a signed, expiring agent-session preview URL) and GET /console/pty/{ticket} (a single-use ticket minted by an authenticated request). Absent or empty auth.tokens (the default) leaves the API exactly as open as the pre-auth MVP — every request is treated as admin — but only on a loopback listener with no OIDC configured: auth.require = true, a non-loopback listen or OIDC make authentication mandatory, and fluxctl serve refuses to start in that case with no token and no OIDC. Two roles:

  • admin — everything: create/stop/pause/resume/exec/delete/build-image/resources/hotplug/freeze/thaw.
  • read-only — any GET route (/v1/vms, /v1/vms/{uuid}, /metrics, /frozen, /stats, /pressure, pool list/get) only; any mutating route (including resources/hotplug/freeze/thaw) returns 403.

POST /v1/egress/check is admin-only despite looking like a read-only diagnostic. It returns {"allow": bool, "reason": "..."} for a host. A matching [sandbox] credential_vault inject_authorization secret is not echoed back: the field is skip_serializing, and its Debug output is [redacted], so the response cannot leak it. The route stays admin-only because it reveals the egress allowlist decision for arbitrary hosts, and an earlier version had no role check at all (see CHANGELOG). Per-sandbox credentials use a separate mechanism, see credential-broker.md.

Optional per-token tenant is authoritative on create (inherited when the body omits it; mismatch → 403) and scopes list/get/mutate to that tenant. List with GET /v1/vms?tenant=acme.

Three credential types are accepted, in this order: a static bearer token, an agent-session token (fxst_..., confined to one session's routes), then an OIDC JWT when auth.oidc_issuer and auth.oidc_audience are both set (role, tenant and actor come from claims). With no Authorization header, a client certificate verified by tls.client_ca supplies the identity (CN, OU=fluxvm-role=admin, OU=fluxvm-tenant=<name>), and the X-Client-Cert-* headers are read only with tls.trust_proxy_headers = true (since 0.4.21). Details, claim mapping and the upgrade notes: auth-oidc-mtls.md.

The same tenant scoping applies to /v1/sandboxes (a sandbox is a VmRecord like any other): POST /v1/sandboxes inherits/enforces the caller's token tenant on the resolved spec exactly like POST /v1/vms does (a template's own tenant, if any, is subject to the same check as an explicit spec.tenant would be), GET /v1/sandboxes only ever returns the caller's own tenant's sandboxes, and every /v1/sandboxes/{id}/... route (snapshot, fs/read, fs/write, process) 404s for a different tenant's token exactly like /v1/vms/{id}/... already did — this was previously missing entirely (see CHANGELOG).

/v1/pools gets the same treatment, name-keyed rather than Uuid-keyed: POST /v1/pools inherits/enforces the caller's token tenant on template.tenant (every member ever backfilled from that pool inherits it unchanged), GET /v1/pools only returns the caller's own tenant's pools, and GET/DELETE /v1/pools/{name}, POST /v1/pools/{name}/claim, and POST /v1/pools/{name}/resize all 404 for a different tenant's token — also previously missing entirely (see CHANGELOG).

POST /v1/pools/{name}/resize (admin-only, body {"size": N}) changes a pool's target size after creation — previously the only way to change a running pool's size at all was to DELETE it (discarding every still-ready warm member) and POST /v1/pools again from the same spec. Growing only updates the stored target and fires the same background backfill create/claim already use (the reaper's own per-tick top-up is the backstop, as always); shrinking deletes excess ready members immediately and synchronously, not on the next reaper tick — a caller asking for a smaller pool is asking to give resources back right away. See docs/operations.md's "Warm VM pools" section for a worked example.

Every pool-returning route (POST/GET /v1/pools, GET/POST .../resize /v1/pools/{name}) returns a PoolView, not the bare stored record: alongside size (the target) and members (ids of members ready right now) it adds ready (members.len(), named explicitly so you don't have to know that's what members counts), pending (size minus ready, floored at 0), and claimed_total (a lifetime count of members this pool has actually handed out via POST .../claim). All three are additive — nothing existing was renamed or removed.

POST /v1/pools/{name}/claim accepts name, ttl_seconds and pod_uid (all optional). pod_uid is the Kubernetes Pod UID a Secure Containers claim is for: it is recorded as the VM's request.pod_uid, so the dataplane attached when the NIC is hot-added mints the Pod's eBPF identity (Pod-scoped network policy, POST /v1/vms/{id}/network/pod-policy) exactly as for a VM created for the Pod. It must be 1–128 characters of [A-Za-z0-9._-] (400 otherwise, checked before a member is taken from the pool).

curl -sS http://127.0.0.1:7788/v1/vms -H 'Authorization: Bearer <token>'
curl -sS 'http://127.0.0.1:7788/v1/vms?tenant=acme' -H 'Authorization: Bearer <token>'

A token listed in [[auth.mfa]] also needs a current TOTP code on guest exec, shells and (optionally) writes. Send it in X-Fluxvm-Totp; a missing or wrong code returns 401, repeated failures 429. See host-hardening.md.

curl -sS http://127.0.0.1:7788/readyz # no token; "ok" requires state_dir (+ dataplane if required)

No token, or a token not in the config, gets 401. A read-only token on a mutating route gets 403. Token comparison is constant-time. Verified on real hardware: 401 with no/wrong token, 200 for read-only on GET /v1/vms, 403 for read-only on POST /v1/vms, 400 for admin on the same route with an invalid body (proving auth let it through to the actual handler), 200 on /healthz and /readyz with no token at all even with auth enabled.

Create through REST:

curl -sS http://127.0.0.1:7788/v1/vms \
-H 'content-type: application/json' \
--data-binary @examples/qemu.json | jq
# Production-shaped example (tenant + tap/netns):
curl -sS http://127.0.0.1:7788/v1/vms \
-H 'content-type: application/json' \
--data-binary @examples/create-vm-prod.json | jq

Whole-stack production checklist: PRODUCTION.md · ./scripts/release-checklist.sh.

Exec through REST (agent.enabled: true required, see operations.md):

curl -sS http://127.0.0.1:7788/v1/vms/<uuid>/agent \
-H 'content-type: application/json' \
-d '{"command": "echo hello", "timeout_seconds": 30}' | jq

VM JSON contract​

backend is one of "qemu", "cloud-hypervisor", "firecracker", or "auto" (see Auto backend selection — the persisted/returned record always shows the resolved concrete backend, never "auto").

{
"name": "job-123",
"tenant": "acme",
"backend": "qemu",
"image": "/var/lib/fluxvm/images/ubuntu.qcow2",
"vcpus": 2,
"memory_mib": 2048,
"disk_size_gib": 20,
"network": {
"mode": "user",
"forwards": [
{"host_port": 2222, "guest_port": 22, "protocol": "tcp"}
]
},
"cloud_init": {
"hostname": "job-123",
"user": "zyvor",
"ssh_authorized_keys": ["ssh-ed25519 AAAA..."],
"packages": ["curl"],
"runcmd": ["echo hello > /tmp/hello"]
},
"agent": {"enabled": true, "port": 17777},
"ttl_seconds": 600,
"extra_args": [],
"storage": "default"
}

Optional tenant is a first-class string for multi-team hosts (GET /v1/vms?tenant=). See examples/create-vm-prod.json.

Optional data_disks (QEMU only), e.g. [{"name": "disk1", "backing": "/var/lib/fluxvm/images/imported/web01/disk1.raw"}], creates each disk as a per-VM qcow2 overlay on backing before the first boot. backing is checked like a POST /v1/vms/{uuid}/disks path and never written to. An image import's suggested body fills it for every disk after the boot disk.

QEMU disk I/O. A data disk's io (in data_disks or on POST /v1/vms/{uuid}/disks) and the create body's root_disk_io take the same object; every field is optional:

{"bus": "virtio-blk", "aio": "io_uring", "cache": "none", "iothread": true, "queues": 4}
  • bus: scsi (default for data disks), virtio-blk, or vhost-user-blk. The root disk is always virtio-blk, so root_disk_io.bus must be unset or virtio-blk.
  • aio: io_uring, native (needs cache none or directsync) or threads.
  • cache: none, writeback (QEMU's default), writethrough, directsync or unsafe.
  • iothread: a dedicated QEMU iothread for the disk (virtio buses).
  • queues: virtio queues, 1-64 (virtio buses).
  • throttle: IOPS and bandwidth limits QEMU enforces (block_set_io_throttle) on scsi, virtio-blk and root disks: iops_total, iops_read, iops_write, bps_total, bps_read, bps_write (bytes/s), plus iops_total_max/bps_total_max bursts lasting burst_seconds (default 1). A *_total limit excludes its read/write pair. Limits are set right after QEMU starts (including on a migration target, since QEMU does not migrate them; a failure stops the launch) and when a disk is hot-added. PUT /v1/vms/{uuid}/disks/{name}/throttle changes them live and keeps them for later starts. A root throttle alone keeps the root's if=virtio drive, so adding one does not move the guest's PCI layout. GET .../disks reports the configured throttle. vhost-user-blk disks are refused: their I/O does not pass QEMU's block layer.

A vhost-user-blk disk is opened by a per-disk qemu-storage-daemon, not by QEMU. Its queues and iothread are the daemon's. Hot-adding one needs shareable guest memory, so create the VM with "shared_memory": true. A stopped VM gets it at the next boot either way. These disks carry no serial number. Live resize and hibernation refuse them. CBT backups and live backups leave them out; CBT lists them as skipped.

Hot-added virtio-blk and vhost-user-blk disks go onto PCIe root ports reserved when the VM is created: disk_hotplug_ports (default 4, at most 16). VMs created before this field existed have none, so their device layout, live migration and saved state are unchanged. They can still attach virtio disks while stopped. SCSI disks need no port.

The QEMU options are stored next to the disk as disks/<name>.io.json, so a disk keeps them across reboots. GET /v1/capabilities reports host.io_uring (the kernel allows io_uring), host.vhost_user_blk (qemu-storage-daemon is installed) and host.nvme_multipath (native NVMe multipath is on, so an NVMe-oF namespace with several paths is a single /dev/nvmeXnY).

A disk on a host block device (a path under /dev, or a storage: shared root on one) is added to the VM's device-cgroup allowlist on the eBPF dataplane. This happens at boot, and also before a live attach. The allowlist holds 8 entries: /dev/kvm, /dev/vhost-vsock, /dev/net/tun, one per VFIO group, then one per block device. An attach that would overflow it is refused.

Optional cdroms (QEMU only, at most 4), e.g. [{"name": "install", "path": "/var/lib/fluxvm/images/win2022.iso"}, {"name": "virtio", "path": "/var/lib/fluxvm/images/virtio-win.iso"}], attaches each ISO read-only as a SATA CD-ROM on the q35 AHCI controller (ide.0, ide.1, …), so Windows Setup sees it with no extra drivers. path is checked like a data disk backing. No boot order is forced: firmware skips a blank root disk and boots the first CD-ROM, and once an OS is installed it boots from the disk. To install onto a blank disk, point image at an empty raw or qcow2 file and set disk_size_gib.

Ejecting install media. A VM with media in a CD-ROM can't be live-migrated, because the ISO path is host-local. Once the OS is installed, eject the medium:

curl -X POST -H "Authorization: Bearer $TOKEN" \
http://127.0.0.1:7788/v1/vms/$ID/cdroms/install/eject
  • Needs an admin-role token and is tenant-scoped like every /v1/vms/{uuid} route (a tenant token can eject only its own tenant's VMs).
  • If the VM is running on QEMU, the medium is removed live over QMP: blockdev-open-tray (forced, so a guest lock doesn't block it), blockdev-remove-medium, blockdev-close-tray, then the read-only cd-<name> block node is deleted. The guest sees an empty drive. Nothing is hot-unplugged.
  • The drive is recorded as empty ("path": "" in cdroms[]). Later starts, restarts and migration receivers create a bare ide-cd device with no medium, so the guest's device layout (and Windows drive letters) stays the same.
  • Ejecting an already-empty drive succeeds and changes nothing. An unknown name returns 400 cdrom "<name>" not found.
  • The response is the updated VM record. The action is audited as vm.cdrom.eject.
  • Creating a VM with an empty path is still rejected (cdrom "<name>" needs a path). An empty drive comes only from an eject.

Live migration rejects only CD-ROMs that still hold media, with eject cdrom "<name>" first (POST /v1/vms/{id}/cdroms/{name}/eject). Inserting new media into an empty drive isn't supported yet; recreate the VM with the ISO instead.

agent.enabled turns on the vsock guest agent (fluxctl exec) for this VM — the guest image must have fluxvm-guest-agent installed and enabled (see the Quick start section and build-image-tutorials.md). agent.port is the AF_VSOCK port the guest listens on (not a host TCP port); it defaults to 17777 and rarely needs changing, since each VM already gets its own host-unique vsock CID.

storage is one of "default" (the implicit default when the field is omitted entirely — qcow2/raw, exactly as before this existed), "lvm-thin", "nbd", "ceph-rbd", or "ceph-rbd-in-place" — see Storage backends.

Networking modes​

none:

{"mode":"none"}

QEMU user networking:

{
"mode":"user",
"forwards":[{"host_port":2222,"guest_port":22,"protocol":"tcp"}]
}

TAP/bridge (all VMMs):

{
"mode":"tap",
"bridge":"vmbr0",
"mac":"06:00:AC:10:00:02"
}

When tap_name is omitted, the manager creates one from the VM UUID.

macvtap (QEMU and Cloud Hypervisor only — see below):

{
"mode": "macvtap",
"parent": "eth0",
"macvtap_mode": "bridge",
"mac": "52:54:00:aa:bb:cc"
}

Gives the VM its own MAC directly on parent's link — no host bridge involved. macvtap_mode is the macvtap link mode: bridge (default — siblings on the same parent can reach each other, but not the parent itself directly), vepa, private, or passthru. The manager creates a per-VM macvtap device on parent, opens its /dev/tapN character device, and passes that file descriptor directly to the VMM (-netdev tap,fd=N for QEMU, --net fd=N for Cloud Hypervisor) — there's no persistent named tap the VMM opens itself, which is why Firecracker doesn't support this mode: its API only accepts a host device name it opens via /dev/net/tun, with no fd-passing option.

Direct — bridge-less tap (opt-in; direct-datapath.md):

{
"mode": "tap",
"mac": "02:00:00:00:0a:0a",
"direct": {"outer": "enp1s0", "mode": "l2-uplink", "guest_ips": ["192.168.1.50"]}
}

An eBPF redirect pairs the VM's tap with outer instead of a bridge. mode is l2-uplink (a physical NIC that is not a bridge/bond port; frames are steered by destination MAC, and ARP requests by the declared guest_ips) or peer-veth (a Pod's veth, with netns_path naming the Pod network namespace the tap is created in; used by the Secure Containers shim). bridge, netns: true and extra NICs cannot be combined with direct. It requires sandbox.dataplane.mode = "ebpf" or "cilium" (legacy is rejected), a failed dataplane attach fails the create, and the tap is handed to QEMU or Cloud Hypervisor (by name, or as a descriptor when it lives in another namespace); Firecracker and the native hypervisor accept only the host-namespace form because they cannot take a descriptor. Policy.allowed_network_modes sees it as tap.

Virtual networks (Linux)​

A tap NIC with vnet joins an eBPF-routed virtual network instead of a bridge:

{"mode":"tap","mac":"52:54:00:12:34:56","vnet":{"vni":100,"ip":"10.50.0.11","prefix_len":24,"gateway":"10.50.0.1"}}

mac is required; bridge, netns, direct and extra are refused, and NIC hotplug is refused on such VMs. The network itself (CIDR, gateway, endpoints on every host, uplink, SNAT address, floating IPs) is applied with PUT /v1/network/vnets/{vni}. Capability host.vnet. See virtual-network.md.

NIC hotplug (QEMU)​

POST /v1/vms/{uuid}/hotplug/nic (admin) attaches a NIC to a running VM, used by Secure Containers after a warm-pool claim (the pool template boots with network.mode=none). On a netns: true VM, QEMU runs inside the namespace, so the daemon opens the new bridge tap and passes it over QMP (getfd); on relaunch every extra NIC's tap is inherited as -netdev tap,fd=. Send either a bridged NIC:

{"bridge": "fvbhab12cd", "mac": "02:00:00:00:00:01"}

or a bridge-less one (the daemon creates the tap, applies the dataplane, then passes the tap to QEMU over QMP with getfd + netdev_add fd= on one session):

{"mac": "02:00:00:00:00:01",
"direct": {"outer": "eth0", "netns_path": "/run/netns/fvcni-ab12cd", "mode": "peer-veth"}}

bridge and direct are mutually exclusive (400 otherwise). A direct hotplug needs a VM booted with network.mode=none; on any failure the daemon removes the tap and dataplane it created.

POST /v1/vms/{uuid}/hotplug/nic/unplug (admin) removes an extra bridged NIC from a running VM, named by {"mac": "..."} or {"tap": "..."}, and deletes its tap. The guest has 10 s to release the device. The primary NIC can't be hot-removed.

Share hotplug (QEMU)​

POST /v1/vms/{uuid}/hotplug/share (admin) hot-adds a virtiofs share to a running VM, so a warm-pool member (which keeps its template's shares) can receive a Pod's rootfs and volumes after a claim:

{"host_path": "/var/lib/fluxvm/some/dir", "read_only": false}

The reply is {"tag": "fs<N>"}, N being the share's index in the VM's shared_folders; mount it in the guest with mount -t virtiofs fs<N> <dir>. The path must be absolute, without .., and a directory the daemon can see (a sandboxed daemon with PrivateTmp does not see /tmp). The VM must have been created with "shared_memory": true (vhost-user-fs needs shareable guest memory); up to four shares can be hot-added (reserved root ports). On failure the virtiofsd is killed and nothing is recorded.

Speculation, grants, exec policy and memory density​

These routes came with the crash-safe lifecycle work. They are implemented and unit tested; none has been run end-to-end on real VMs yet (each guide has a status table). The OpenAPI document served at GET /v1/openapi.json is generated by openapi::spec() in crates/fluxvm-api/src/openapi.rs; it has not been updated for these routes, headers or query parameters, and there is no hand-maintained spec file. Until it is, this page is the contract.

Every route here needs an admin token except GET /v1/vms/{uuid}/balloon and GET /v1/vms/{uuid}/memory. They are tenant-scoped like the rest of /v1/vms/{uuid} and /v1/sandboxes/{id}.

Speculation and changesets​

Guide: speculative-execution.md.

POST /v1/sandboxes/{id}/speculate runs a command in an isolated copy and returns a pending changeset. The sandbox is started on demand like other sandbox routes.

{"command": "make build", "timeout_seconds": 120, "paths": ["/work"], "ttl_seconds": 600}
  • command (required).
  • timeout_seconds (optional) bounds the command.
  • paths (optional for procbox sandboxes, default ["/"]; required for VM sandboxes, 400 otherwise) limits the diff to those directories.
  • ttl_seconds (optional) is how long the changeset stays decidable: default 3600, clamped to 1..=86400.

The response is a changeset. The same shape is returned by GET .../changesets/{cs} and by approve, reject and apply:

{
"id": "5b0f6c1e-0f0e-4a63-9f58-0b2f1e0f7a11",
"sandbox_id": "0c1d2e3f-0000-4000-8000-000000000001",
"state": "pending",
"created_at": 1790000000,
"updated_at": 1790000000,
"expires_at": 1790000600,
"command": "make build",
"exit_code": 0,
"stdout": "...",
"stderr": "",
"paths": ["/work"],
"run_via": "fork",
"changes": {"added": ["/work/out.bin"], "modified": [], "deleted": [], "unchanged": 41},
"side_effects": {
"egress": "blocked",
"destinations": [],
"replayable": ["1 file change(s) under /work"],
"non_replayable": []
},
"base_files": 42,
"staged": {"/work/out.bin": {"blob": "000000", "mode": 33188, "size": 1048576}},
"unstaged": [],
"apply_started_at": null,
"error": null
}

Timestamps are Unix seconds. run_via is workspace-copy (procbox), fork or snapshot (VM sandboxes). side_effects.egress is blocked, allow_listed or unrestricted, and is declared, not enforced (see the guide). stdout and stderr are capped at 64 KiB each, with a [truncated N bytes] marker.

  • GET /v1/sandboxes/{id}/changesets returns {"items": [changeset, ...]}, newest first.
  • POST .../changesets/{cs}/approve and .../reject decide a pending changeset; no body.
  • POST .../changesets/{cs}/apply writes an approved changeset's file changes into the real sandbox.

Status codes: 404 unknown changeset (or one that belongs to another sandbox); 409 invalid state transition, conflict (the real files changed since the changeset's base; nothing is written), expired, or another request is already working on that changeset; 422 the changeset cannot be applied (for example files that could not be staged); 403 a non-admin token.

Credential grants​

Guide: credential-broker.md.

POST /v1/sandboxes/{id}/grants returns 201:

{"secret_ref": "github-token", "hosts": ["api.github.com"], "ttl_seconds": 900}
  • secret_ref (required, 1-128 chars). Without value, the daemon reads FLUXVM_SECRET_GITHUB_TOKEN from its own environment.
  • value (optional, write-only): the full header value, for example "Bearer abc". Never returned.
  • hosts (required, at least one): bare host names, exact or domain suffix. *, /, :, @, ? and whitespace are rejected.
  • ttl_seconds (1..=86400, default 3600) or expires_at (RFC 3339, at most 24 h ahead). If both are sent, expires_at wins.
  • At most 32 live grants per sandbox.

Response (GrantInfo, which has no secret field):

{
"id": "7e0f0c52-5d57-4b35-8d3f-6a1a3d2b9c10",
"sandbox_id": "0c1d2e3f-0000-4000-8000-000000000001",
"secret_ref": "github-token",
"hosts": ["api.github.com"],
"created_at": "2026-10-08T12:00:00Z",
"expires_at": "2026-10-08T12:15:00Z",
"bound": true
}

bound: false means the sandbox has no known guest IP, so the grant cannot match any traffic.

  • GET /v1/sandboxes/{id}/grants returns {"grants": [GrantInfo, ...]} (live grants only).
  • DELETE /v1/sandboxes/{id}/grants revokes all and returns {"revoked": true|false}.
  • DELETE /v1/sandboxes/{id}/grants/{grant_id} revokes one and returns {"revoked": true}, or 404.

Every verb is admin-only, GET included. Validation failures (bad host, bad TTL, unresolvable secret_ref, more than 32 grants) return 400.

Host overview and capabilities (macOS)​

Two read-only routes feed a dashboard without scraping /metrics:

  • GET /v1/host/history[?since=<rfc3339>]: the last hour of host memory and per-VM runner usage at 5-second spacing: {interval_seconds, capacity, items: [{ts, memory_available_mib, memory_total_mib, memory_pressure, vms: [{vm_id, cpu_seconds_total, rss_bytes}]}]}, oldest first, since exclusive. In memory only: it is empty after a daemon restart. A dashboard uses it to draw full sparklines on connect.
  • The vm.fork and vm.branch.start events (GET /v1/events?event=vm.) carry total_ms and report, a JSON array with one entry per child: {id, name, prepare_ms, restore_ms, identity_ms, identity_via, hooks_ms, ready_ms, total_ms}.
  • POST /v1/vms/{id}/fork/plan and POST /v1/branches/{id}/start/plan (admin; the same body as the real call): what the request would be refused for and what it is expected to cost, without doing it. {ok, count, parallelism, refusals: [{code, reason}], predicted: {extra_rss_mib_low, extra_rss_mib_high, extra_rss_basis, disk_logical_bytes_each, est_total_ms, est_basis, host_available_mib, host_available_after_low_mib, host_available_after_high_mib}}. Refusals come from the checks the real call runs (count, parallelism, hooks, forkable, network, quota, name), all collected. The prediction is a bound, not a promise: low is the per-child resident size: the larger of this source's running fork children's average and the source's own (young children understate it), else a 256 MiB fresh-restore floor; high is every child using all its memory. Children fault pages back in after a restore, so a child's size grows towards (and on the M4 slightly past) its parent's. The time is the median of forks recorded on this Mac and is absent until there is one.
  • GET /v1/vms/{id}/usernet/policy and PUT /v1/vms/{id}/usernet/policy (admin): the egress policy of an apple.usernet guest. PUT takes any of allow_cidrs, deny_cidrs, allow_ports, deny_ports, dns_allow, dns_deny, allow_private, allow_cidrs6, deny_cidrs6 (absent keeps, a present empty list clears), validates them like create, stores them in the VM record (so a restart, a fork or a branch start uses them) and, when the gateway runs, applies them at once: the daemon rewrites the gateway's config and signals it, and new connections and DNS queries follow the new policy while open connections are not cut. Reply: the policy plus applied and a note. Port forwards, MTU and addresses are not changed this way.
  • GET /v1/host/images: {"builtin_cached": ["debian-13", ...]}, the built-in images with a ready copy on this host (no network access). fluxvm-agent node sends it in the heartbeat and fluxvm-agent central prefers a Mac that already has the image.
  • GET /v1/host/storage: where disk space goes. volume (path, total_bytes, free_bytes of the filesystem holding the state directory), vms[] (vm_id, name, status, disk and other as {allocated_bytes, logical_bytes, files}: the root disk, and everything else in the workspace), branches[] (id, name, usage), images (the image cache) and state_dir (everything). Allocated is the blocks a file really occupies (a sparse 3 GiB disk holding 700 MiB counts 700 MiB), but copy-on-write clones (fork, branches, vm clone) share blocks and each clone reports them in full, so the parts can add up to more than the volume holds. volume.free_bytes is the truth about what is left.
  • GET /v1/host/overview: daemon (version, uptime_seconds), host (os_version, cpu_count, memory_total_mib, memory_available_mib, memory_pressure normal|warn|critical, nested_virtualization, rosetta), vms (total, by_status, by_backend), branches, usage[], helpers (runner, switch, blockd, usernet present or not) and networking (vm_networking_entitlement, netobs_enabled). usage[] has one row per running or paused vz VM: vm_id, name, status, cpu_seconds_total (cumulative: take two samples for a rate), rss_bytes, service_attributed and started_at (Unix seconds, the runner process start). CPU and RSS are the runner plus its Virtualization XPC service when service_attributed is true, otherwise the runner alone (the guest's own usage is not included). Verified against ps on an M4: started_at exact, RSS equal to runner plus service.
  • GET /v1/host/apple/capabilities: the capability table judged against this Mac: per feature supported, note, min_macos, available_here (null when the host has not been probed) and reason (macOS version, chip generation, missing com.apple.vm.networking, missing Rosetta).

Node stats​

GET /v1/stats is one cheap sample of this host and every VM on it, for the fleet's metrics collector (fleet-metrics.md); anything that moves is a cumulative counter, so a reader takes two samples and divides by the time between them. Gauges (memory, disk free) are as of now, and a figure this host cannot measure is null, never a made-up zero.

  • host: cpus, cpu_busy_ticks and cpu_total_ticks (only the ratio of two samples' differences means anything), memory_total_mib, memory_available_mib, disk_total_bytes and disk_free_bytes of the volume holding the state directory, vm_count, vms_running.
  • ts is the sample time. Per VM: id, name, status, backend, vcpus, memory_mib, cpu_seconds_total, memory_used_bytes, disk_read_bytes_total, disk_write_bytes_total, net_rx_bytes_total, net_tx_bytes_total and a source that says which of these it can supply. vz-process (a vz guest, read from the runner plus its Virtualization XPC service: CPU, resident memory, disk bytes, no network because Virtualization.framework has no per-VM counters), vz-runner (the XPC service could not be attributed, so the runner process only), cgroup (a Linux backend, from the VM's cgroup v2 and its TAP device) or none (stopped, or nothing readable).

Volumes and tunnels (fleet platform)​

These routes are the node half of fleet volumes and tunnels. They are admin-only: tenants reach them through fluxvm-agent central, which authenticates the tenant, applies quotas and policy, and calls them with the fleet token.

Volumes (/v1/volumes) are sparse raw disk files under <state_dir>/volumes/ that outlive the VMs using them. A volume is {name, path, format: "raw", size_gib, size_bytes, allocated_bytes, attached_to: [vm ids]}.

  • POST takes {"name", "size_gib"}. A name is 1 to 63 lowercase letters, digits and -, starting with a letter or digit (it becomes a file name); size_gib is 1 to 16384. At most 1024 volumes per host.
  • A VM uses a volume as an apple.extra_disks image disk (path is what it names). attached_to lists the VM records that reference the file; DELETE and resize answer 409 while it is not empty. A resize only grows.
  • Deleting the VM leaves the volume alone: that is the point of it.

Tunnels (GET /v1/vms/{uuid}/tunnel?port=N) upgrade to a WebSocket that carries one TCP connection to port N of a running VM; each binary message is raw bytes (at most 64 KiB from the guest, 256 KiB from the caller), and each direction waits for its write, so a slow end slows the other. The node connects only to the VM:

  • an apple.usernet VM: the host listener of its own TCP port forward for that guest port (409 when it publishes none; add one to apple.usernet.port_forwards);
  • a tap + netns VM: from inside its network namespace, to the address its dnsmasq leased (link-local 169.254.x.y is allowed there; the cloud metadata address is not);
  • a vz VM on the shared network: to the address the guest reported on its serial console, only if it lies on this Mac's own VM network (a bridge* interface's subnet) and no other VM on the node reports the same address. The address comes from the guest, so anything outside that (the Mac's LAN, another network) is refused with 409.

Loopback, unspecified, multicast and the node's own interface addresses are always refused, so a tunnel never reaches the node's own services. The VM must be running (409 otherwise) and the port 1 to 65535; central applies the tenant's port policy before it connects here.

Host topology and NUMA binding (Linux)​

  • GET /v1/host/topology: NUMA nodes (CPUs, thread siblings, memory, hugepage pools, distances) and placeable PCI devices (GPU, NIC, NVMe, accelerator) with NUMA node, PCIe root, IOMMU group and SR-IOV VFs.
  • POST /v1/host/pci/bind / POST /v1/host/pci/release: /v1/host/gpus/bind|release for any PCI function; admin.
  • CreateVmRequest.numa binds guest NUMA nodes to host nodes (QEMU only). See host-topology.md.
  • GET /v1/host/mdev: mediated device parents, their profiles (free instances, framebuffer, MIG slices) and existing instances. POST /v1/host/mdev (admin, {"parent", "type", "uuid"?}) creates an instance (201; 409 for a UUID on another parent or type, 503 when the profile has none free). DELETE /v1/host/mdev/{uuid} (admin, 204) refuses an instance a VM lists or holds. vfio_devices takes instance UUIDs. See vgpu.md.
  • GET /v1/host/gpus/health: per GPU healthy, degraded or unhealthy with reasons, AER counters, PCIe link and nvidia-smi counters when installed.
  • GET /v1/host/infer: whether the host OpenAI-compatible model server answers on [apple].host_infer_port for apple.host_gpu sandboxes (see host-inference.md). fluxctl infer status / infer serve --stand-in.

Balloon and memory​

Guide: memory-density.md.

GET /v1/vms/{uuid}/balloon, and POST /v1/vms/{uuid}/balloon (admin) with {"balloon_mib": 256} (0 deflates), both return:

{"memory_mib": 1024, "target_mib": 256, "actual_mib": 192}

target_mib is what was requested; actual_mib is what the guest driver has reached so far. A running VM on the flux-vm backend's KVM engine, the vz backend, or the qemu backend created with balloon has a balloon; anything else returns 400, as does a balloon that would leave the guest under 64 MiB. For QEMU, memory_mib includes hot-added DIMMs.

CreateVmRequest.balloon (QEMU only) adds a virtio-balloon-pci device:

{"balloon": {"free_page_reporting": true, "deflate_on_oom": true, "stats_interval_secs": 5}}

All three fields are optional and default to the values shown. balloon is refused with hugepages, vfio_devices and confidential security profiles.

GET /v1/host/memory samples the host now:

{"mem_available_mib": 20480, "mem_total_mib": 31744, "psi_some_avg10": 0.4, "psi_full_avg10": 0.0}

A field is null when the host does not expose it (no PSI in the kernel). host.balloon in GET /v1/runtime/capabilities is true when both routes are available.

GET /v1/host/pressure samples whole-host PSI from /proc/pressure now, in the same shape as GET /v1/vms/{uuid}/pressure:

{
"cpu_some": {"avg10": 20.78, "avg60": 19.86, "avg300": 18.66, "total": 76804200790},
"memory_some": {"avg10": 0.0, "avg60": 0.0, "avg300": 0.0, "total": 6194},
"memory_full": {"avg10": 0.0, "avg60": 0.0, "avg300": 0.0, "total": 6164},
"io_some": {"avg10": 40.67, "avg60": 30.67, "avg300": 25.89, "total": 286750114310},
"io_full": {"avg10": 22.75, "avg60": 15.68, "avg300": 14.5, "total": 220501129495}
}

Averages are percentages of wall time some (or all) tasks stalled on the resource; total is microseconds. A field is null when the kernel has no PSI. Kairon's kairon-node publishes these on its Node for DRS.

GET /v1/vms/{uuid}/memory:

{
"vm_id": "0c1d2e3f-0000-4000-8000-000000000001",
"configured_mib": 1024,
"usage": {"rss_kib": 310000, "pss_kib": 180000, "private_kib": 120000, "shared_kib": 190000, "swap_kib": 0},
"balloon": {"memory_mib": 1024, "target_mib": 0, "actual_mib": 0}
}

usage is null when the VM has no VMM process or /proc/<pid>/smaps_rollup is unreadable; balloon is null when the VM has no balloon. A ballooned QEMU VM also reports guest (total_mib, free_mib, available_mib, disk_caches_mib, last_update) once the guest driver has sent statistics; a statistic the guest did not send is null.

First-command latency: ?ready=exec​

POST /v1/vms, POST /v1/vms/{uuid}/fork and POST /v1/pools/{name}/claim accept ?ready=exec (or ?wait_first_command=true). The call then blocks until a trivial guest-agent exec (true) succeeds, which needs the guest agent enabled, and adds timings measured from the start of the request. For create and claim they are merged into the VM record:

{"id": "...", "first_command_ms": 412, "phases": {"create_done_ms": 180, "agent_wait_ms": 231, "first_exec_ms": 412}}

create_done_ms is when create or claim returned, agent_wait_ms the extra wait for the first successful exec, and first_exec_ms the total. For fork, the body keeps items and elapsed_ms and adds first_command_ms (the slowest child) and first_command (one {first_command_ms, phases} object per child). A vm.first_command event is recorded for each VM. scripts/bench-first-command.sh uses it.

Errors​

Every error is JSON: {"error": "<message>", "code": "<CODE>"}. Branch on code (it does not change); the message is for people and may be reworded. The code follows the HTTP status:

StatuscodeTypical cause
400BAD_REQUESTMalformed body, query or Idempotency-Key
401UNAUTHORIZEDMissing or invalid credential: a bearer token, or (with mTLS) a client certificate with no CN
403FORBIDDENRole or session token does not allow the route
404NOT_FOUNDNo such VM or session, or one that belongs to another tenant (never 403, so ids do not leak)
409CONFLICTState conflict: VM not running, a duplicate in flight, a changeset whose files moved
412PRECONDITION_FAILEDIf-Match no longer matches
413PAYLOAD_TOO_LARGERequest body over the limit
422UNPROCESSABLESame Idempotency-Key reused with a different request
428PRECONDITION_REQUIREDThe request needs a condition first (an operator approval, an If-Match)
429RATE_LIMITEDHonour Retry-After
501NOT_IMPLEMENTEDThe backend does not support the route
503UNAVAILABLECapacity (for example no free GPU); retry later
other 5xxINTERNALServer fault

Two special codes ride on the generic statuses: APPROVAL_REQUIRED (a 428 that also carries approval_id) and STEP_UP_REQUIRED (a 401 asking for a TOTP code). Tunnel, idempotency and fleet-central errors send a code too. The fleet central's sign-up and SSO browser flows still send only error; treat a missing code as unknown and fall back to the status.

Not sure what a code means or what to do about it? fluxctl explain answers offline, from the same table:

fluxctl explain RATE_LIMITED # meaning, usual causes, what to do (case, - and _ do not matter)
fluxctl explain 428 # every code on an HTTP status
fluxctl explain required # part of a code name
fluxctl explain --list [--json] # all of them

It exits 1 for an unknown code (with a did you mean). The table is fluxvm_core::error_codes; a test in fluxvm-api fails when the API can send a code that the table does not explain.

Idempotency-Key​

POST /v1/vms, DELETE /v1/vms/{uuid}, POST /v1/vms/{uuid}/snapshot and POST /v1/vms/{uuid}/fork honor an Idempotency-Key request header (1-255 visible ASCII characters, else 400). Other routes ignore it, and requests without it behave as before.

  • The key is scoped by token tenant (or caller) and by method plus path.
  • The first 2xx response is fsynced under <state_dir>/idempotency before it is returned. A retry with the same key and an identical method, path and body gets the stored status and body back with the response header Idempotent-Replayed: true, and nothing runs again.
  • Same key with a different method, path or body: 422.
  • Same key while the first request is still running: 409.
  • Only 2xx responses are stored, so a failed attempt can be retried under the same key.
  • Records expire after 24 hours. Request bodies over 16 MiB get 413. A response over 8 MiB returns 500 and is not stored.
  • The tenant guard runs before a stored response is replayed.

Exec policy and enforcement​

Guide: guest-exec-policy.md. POST /v1/vms/{uuid}/agent and POST /v1/sandboxes/{id}/process accept an optional policy object (the procbox policy shape) next to command and timeout_seconds. With a policy, the guest confines the command with Landlock and seccomp, and the exec response carries enforcement:

{"command": "ls /usr", "policy": {"read": ["/bin", "/usr", "/lib", "/lib64"]}}
{"result": "exec", "exit_code": 0, "stdout": "...", "stderr": "", "enforcement": {"landlock_abi": 3, "filesystem": true, "not_enforced": []}}

If the policy cannot be enforced, the command is not run and the response is an error. Procbox sandboxes reject a per-exec policy (400). Without policy, the response is unchanged and has no enforcement key.

  • GET /v1/vms/{id}/guard (read-only): the VMM Guard state for a VM — mode, exec allow-list, applied rule packs and expired_packs. See docs/host-hardening.md.