User guide: sandboxes (spec.sandbox)
Run a short-lived, ephemeral workload -- e.g. an AI agent's own generated code -- on FluxVM's own lightweight, fast-boot in-tree hypervisor track, instead of a full QEMU/Cloud Hypervisor/Firecracker VM. Admin-only, API-only first cut, no dashboard yet.
What this is
apiVersion: kairon.zyvor.dev/v1
kind: Machine
metadata: {name: agent-run}
spec:
image: {path: /images/sandbox.qcow2}
resources: {cpu: "1", memory: 512Mi}
ttlSeconds: 900 # auto-expire after 15 minutes
guestAgent: {console: true} # unlocks Kairon's own file/exec/console UI (see below)
sandbox: {} # opts into FluxVM's sandbox track
Or, booting from a pre-built template instead of an image (faster boot,
no spec.image/spec.resources needed -- FluxVM loads the template's own
pre-baked spec and ignores both entirely):
spec:
sandbox:
templateName: alpine-agent
Under the hood: kairon-node calls FluxVM's real POST /v1/sandboxes
instead of POST /v1/vms -- FluxVM forces the runtime to its own
lightweight flux-vm backend and force-enables its own vsock guest agent
regardless of spec.guestAgent, whatever that says. Everything else about
a sandbox Machine -- status.phase, spec.powerState, spec.ttlSeconds,
deletion -- works exactly like any other Machine.
Building a template first
A template is a pre-exported OCI image rootfs FluxVM can boot from instantly, skipping a full image boot. Build one via the node-scoped admin API (admin-only -- pulling and exporting an arbitrary OCI image is a real resource cost):
curl -X POST https://<kairon-ui>/api/v1/nodes/<node>/templates \
-H "Authorization: Bearer <token>" \
-d '{"name": "alpine-agent", "imageRef": "docker.io/library/alpine:3.19"}'
List what's already built on a node with GET /api/v1/nodes/<node>/templates. Templates are per-node state -- one
built on worker-1 isn't visible from worker-2; a spec.sandbox.templateName
referencing a template that doesn't exist on the Machine's assigned node
fails with FluxVM's own clear error.
Guest access
FluxVM always runs its own vsock guest agent for a sandbox, but Kairon's
own guest file access and
backend-agnostic guest exec
still gate on spec.guestAgent.console: true client-side regardless --
set it anyway to use those against a sandbox. The QEMU-only
guest exec (qemu-guest-agent) and
interactive text console work the same way
too, since both ride channels a sandbox's lightweight hypervisor actually
supports (the vsock agent) or don't need QEMU specifically.
HTTP proxy: reaching a service running inside the sandbox
ANY /api/v1/machines/{ns}/{name}/sandbox-http/{port}/{path...} relays
an arbitrary HTTP request straight through to a sandbox's own guest HTTP
server on port -- method, headers, body, and the guest's response
status/headers/body all pass through. Useful for a webhook receiver or dev
server the sandboxed workload starts on its own. Admin-only: this is a
materially broader capability than guest exec or file access -- whatever
the guest's own web server does with an arbitrary request is between the
guest and the caller, Kairon has no visibility into it.
Requires the Machine to actually have a guest IP (spec.network using
tap/bridge networking, not user/SLIRP -- the proxy dials the guest
directly, the same requirement FluxVM's own proxy route has).
curl -X POST https://<kairon-ui>/api/v1/machines/default/agent-run/sandbox-http/8080/api/webhook \
-H "Authorization: Bearer <token>" \
-d '{"event": "..."}'
Listing sandboxes
GET /api/v1/nodes/{node}/sandboxes lists every FluxVM sandbox on a
node -- node-scoped, not Machine-scoped, since it reflects FluxVM's own
real state (including a sandbox created directly against FluxVM outside
Kairon entirely), the same posture GET /api/v1/nodes itself has for
node visibility. Any authenticated operator, since this is read-only.
Checking egress policy
POST /api/v1/nodes/{node}/egress-check (admin-only, body {"host"})
asks a node whether a sandbox's outbound request to host would be
allowed -- FluxVM's own stateless check against that node's static
[sandbox] config (egress_allow_domains, an empty list meaning "allow
everything"). Response: {"allow", "reason", "wouldInjectCredential"}.
There is deliberately no way to change the allowlist through this or
any Kairon API -- it's read from the node's own FluxVM config file at
daemon startup, not a runtime-mutable resource. And wouldInjectCredential
is a boolean, never the actual credential: FluxVM's own response carries
the literal secret value that would be injected for a matching host
(inject_authorization), and Kairon deliberately never returns that value
to a caller. See SECURITY.md's "Egress check" section for why.
Warm pools
FluxVM keeps size VMs pre-booted and Paused, ready for an instant
claim -- much faster than a cold create, at the cost of keeping those VMs'
resident RAM committed on the node the whole time, whether claimed or not.
Node-scoped admin API, no dashboard yet:
POST /api/v1/nodes/{node}/pools(body{"name", "size", "template"},templatea full FluxVMCreateVmRequest-- the same shape a sandbox's ownspecfield takes) creates a pool. FluxVM immediately starts bootingsizeVMs toPausedin the background; the call itself returns as soon as the pool record exists, not once every member is ready.GET /api/v1/nodes/{node}/pools/GET .../pools/{name}list pools or show one's currentmembers(VM UUIDs, any authenticated operator, read-only).POST .../pools/{name}/claim(body{"name", "ttlSeconds"}, both optional overrides, plus optional{"createMachine", "namespace", "machineName"}-- see below) pops one ready member, resumes it, and returns the now-RunningVM. FluxVM triggers its own background backfill to replace the claimed member; Kairon doesn't wait for that to finish. Fails with a clear error if the pool has no ready members right now (rather than silently falling back to a slow synchronous create) -- retry shortly, or make the pool bigger.DELETE .../pools/{name}(admin-only) deletes the pool and every member VM it currently holds, claimed or not -- a genuinely destructive operation, not just removing the pool's own bookkeeping.
By default, a claimed VM is real FluxVM state, not automatically a
Kairon Machine. Claiming hands back FluxVM's own VM record (UUID, name,
guest_ip, status) directly -- with no createMachine, there is no
automatic step that creates a corresponding Kubernetes Machine object
for it, and managing it as an ordinary Kairon Machine afterward (power
state, disruption budgets, quota accounting, the dashboard) needs a
manual follow-up you do yourself.
Set createMachine: true (plus a required machineName, namespace
defaulting to default) to close that gap for this claim. The response
becomes {"vm": <claimed FluxVM record>, "machine": <created Machine>}
instead of the bare VM record. Under the hood: the claim's own FluxVM-side
name is forced to kairon-<namespace>-<machineName> (overriding any
name you also set), matching exactly what Machine{Name, Namespace}.RuntimeName()
computes -- so kairon-node's existing adoption path (it already falls back
to looking up a runtime by that exact name when a Machine has no
status.runtimeID yet) picks up this precise VM on its very next
reconcile tick, instead of creating a second, duplicate one. The new
Machine's spec is built from the pool's own template (fetched fresh at
claim time, the same spec every member was actually booted from):
image, CPU/memory and their maxCpu/maxMemory hotplug headroom,
backend and kernel, cloud-init (including staticNetwork),
numaNode/cpuSet/hugepages, secureBoot/tpm, the guest agents and
ttlSeconds (the claim's own ttlSeconds wins when set). spec.nodeName
is set to the claimed VM's node, so the scheduler can never place the
Machine elsewhere and boot a duplicate. Template fields with no safe
Machine equivalent -- vfio_devices (a device can back only one VM), the
raw network map (per-VM MACs), shared_folders, data_disks and a
storage override -- are not carried; each one present adds a line to a
"machineWarnings" array in the response. If creating the Machine object itself fails (a namespace/name
conflict, an unreachable API server), the response still reports the
successful claim ("vm" populated, "machine": null, plus a
"machineError" explaining what to do next) -- the VM is real, running
state either way; this never pretends the claim didn't happen just
because the follow-up write did.
Deliberately opt-in, not the new default: warm pools exist for fast,
low-ceremony ephemeral VMs, and a full Machine object drags in
finalizer-gated deletion, quota accounting, and webhook validation that
not every caller wants -- omitting createMachine is byte-for-byte the
same behavior as before this existed.
A note on overlap with FluxVM's own Kubernetes integration: FluxVM
ships its own MicroVMPool CRD (part of fluxvm-microvm, a node-local
controller that reconciles against the exact same /v1/pools API this
section wraps) as part of a separate scheduler-native fleet story
("not KubeVirt -- no live migration/CDI/virtctl"). If your cluster also
runs FluxVM's own fluxvm-microvm operator against the same nodes,
don't manage the same pool name from both -- neither coordinates with
the other, and both would independently create/delete/claim against the
identical underlying FluxVM state.
Real limits today (first cut)
- No VFIO passthrough.
spec.deviceClaimson a sandbox Machine is refused outright at create time with a clear error -- FluxVM's lightweight sandbox hypervisor doesn't support PCI passthrough. - A template-based sandbox ignores
spec.image/spec.resources/spec.runtime.kernelentirely. FluxVM loads the template's own pre-baked spec; setting those fields alongsidetemplateNamehas no effect and isn't rejected as a conflict -- know that a template wins before wondering why a resource change didn't apply. - No admin-only NUMA/CPU-set/hugepage support -- those already require
the
qemubackend elsewhere in this project, and a sandbox always resolves toflux-vm. - No dashboard yet. Every capability here (create, list, HTTP proxy,
template build/list) is API-only --
kaironctlhas no dedicated verbs either. claim'screateMachinedoes not carry VFIO devices, the raw network map, shared folders, data disks or a storage override from the pool template -- they come back asmachineWarnings; add the matching Machine fields (deviceClaims,network,volumes,disks) yourself.- No snapshot listing/deletion for templates, matching VM-state snapshot/restore's own posture for tags -- FluxVM owns storage, Kairon has no prune/list-by-age mechanism.
SnapshotSandbox(FluxVM's ownPOST /v1/sandboxes/{id}/snapshot, which exports a sandbox's rootfs state to an explicit file path) isn't wired to a Kairon-facing route yet --internal/fluxvm.Client.SnapshotSandboxexists but has no admin API/consoleproxy relay in front of it today.