KUBEVIRT
KubeVirt Windows VMs
Provision Windows 11 (and other catalog images) on Kubernetes via KubeVirt. Kryton exposes the same REST API — callers never touch VirtualMachine YAML.
Step-by-step production workflow: USER-GUIDE.md § Production
Prerequisites¶
- Kubernetes cluster with KubeVirt and CDI installed
kubectl,virtctl, andqemu-imgon the operator host- A sysprepped Windows qcow2 golden image (operator-provided; Kryton does not ship media)
- CSI volume snapshots —
rancher.io/local-pathcannot snapshot VM disks. Choose a backend (see STORAGE.md):
# Production / this lab: Rook Ceph RBD on dedicated partition
./scripts/enable-kubevirt-snapshots.sh --storage rook-ceph --device /dev/sdb1
export KRYTON_STORAGE_CLASS=rook-ceph-block
# Alternate lab CSI: Longhorn
./scripts/enable-kubevirt-snapshots.sh --storage longhorn
export KRYTON_STORAGE_CLASS=longhorn
That turns on the KubeVirt Snapshot feature gate, ensures the CSI snapshot controller is present, and installs a CSI StorageClass + VolumeSnapshotClass. Kryton creates VirtualMachineSnapshot / VirtualMachineRestore objects (API, UI, krytonctl snapshot).
Build one with the WinForge-style pipeline — see GOLDEN-IMAGES.md.
Golden images (WinForge pipeline)¶
# 1. Build (dockur → Sysprep → capture)
VERSION=11e ./scripts/build-golden-image.sh
# ... Sysprep in guest ...
VERSION=11e FINALIZE=1 ./scripts/build-golden-image.sh
# 2. Bootstrap CDI DataSource
KRYTON_WINDOWS_IMAGE=./out/windows-11e-golden.qcow2 ./scripts/bootstrap-kubevirt-images.sh
# Or HTTP import if published to object storage:
KRYTON_IMAGE_URL=https://artifacts.example/win11.qcow2 ./scripts/bootstrap-kubevirt-images.sh --http
Full details: GOLDEN-IMAGES.md.
Automated setup (recommended)¶
One command bootstraps the golden image, installs the Kryton API, and creates a Windows 11 VM:
export KRYTON_WINDOWS_IMAGE=/path/to/windows11.qcow2
./scripts/setup-kubevirt.sh
Or via Make:
make setup-kubevirt IMAGE=/path/to/windows11.qcow2
What it does¶
- Creates
kryton-imagesnamespace and auto-provisions project namespaces - Uploads your qcow2 via CDI and creates a
DataSource(windows-11-enterprise) - Installs
krytondas a host systemd service (kryton-kubevirt.service) using your kubeconfig - Calls
POST /api/v1/machinesand polls until the VM is running
In-cluster API (Helm + NodePort)¶
make setup-kubevirt IMAGE=/path/to/windows11.qcow2 ARGS='--helm'
Uses deploy/helm/kryton/values-lab.yaml (auth disabled, NodePort 30088).
Bootstrap images only¶
If you only need CDI DataSources:
# Local upload
KRYTON_WINDOWS_IMAGE=/path/to/windows11.qcow2 ./scripts/bootstrap-kubevirt-images.sh
# HTTP import (published golden image)
KRYTON_IMAGE_URL=https://artifacts.example/win11.qcow2 ./scripts/bootstrap-kubevirt-images.sh --http
Manual API usage¶
After setup, the API listens on :9088 by default (host mode, auto-increments if busy) or NodePort 30088 (Helm lab).
export KRYTON_URL=http://127.0.0.1:9088
krytonctl doctor
krytonctl create --image windows-11-enterprise --cpu 4 --memory 8192 win11-01
krytonctl get <id>
krytonctl list
curl -s -X POST "${KRYTON_URL}/api/v1/machines" \
-H 'Content-Type: application/json' \
-d '{
"project": "default",
"name": "win11-01",
"image": "windows-11-enterprise",
"compute": {"cpu": 4, "memoryMiB": 8192},
"disk": {"sizeGiB": 96}
}' | jq .
Configuration¶
| Variable | Default | Description |
|---|---|---|
KRYTON_PROVIDER |
kubevirt |
Required for this path |
KRYTON_KUBECONFIG |
~/.kube/config |
Kubernetes credentials (host mode) |
KRYTON_IMAGE_NAMESPACE |
kryton-images |
CDI DataSource namespace |
KRYTON_PROJECTS |
default |
Kryton projects → K8s namespaces |
KRYTON_AUTH_MODE |
apikey (prod) / disabled (lab) |
API authentication |
| KRYTON_EVENTS_FILE | (memory only) | Durable JSONL event log path |
| KRYTON_EVENT_WEBHOOK_SECRET | (none) | Webhook HMAC signing secret |
krytonctl doctor verifies KubeVirt connectivity, project namespaces, and that every catalog image has a matching CDI DataSource.
Console¶
KubeVirt machines expose consoleUrl when the guest instance is running. Open it in the dashboard or browser to launch an embedded noVNC session proxied through Kryton (/api/v1/machines/{id}/console). With API-key auth the dashboard first mints a short-lived console ticket, because iframes and websockets cannot send an Authorization header; see AUTH.md.
Linux guests¶
The eleven Linux templates run on KubeVirt from CDI DataSources you import yourself; see LINUX-TEMPLATES.md. Linux VMs use BIOS, VirtIO and a NoCloud seed, and need outbound package access on first boot to install qemu-guest-agent.
Production¶
For production estates use Helm with API-key auth — see DEPLOYMENT.md.
helm upgrade --install kryton ./deploy/helm/kryton -n kryton --create-namespace \
--set auth.mode=apikey \
--set auth.existingSecret=kryton-auth
Troubleshooting¶
| Symptom | Fix |
|---|---|
Missing DataSources in doctor |
Run bootstrap-kubevirt-images.sh |
VM stuck Provisioning |
kubectl -n default get dv,pvc,vm — check CDI import |
kubernetes config error on start |
Set KRYTON_KUBECONFIG or KRYTON_KUBERNETES_ENDPOINT |
| Overlay qcow2 upload fails | Script auto-flattens; ensure qemu-img is installed |
Linux VM runs but never reports an IP; cloud-init hangs on apt/dnf |
Guest DNS is failing. On Cilium with kube-proxy replacement, set bpf-lb-sock-hostns-only: true in cilium-config and restart the Cilium agents so VM masquerade traffic reaches the DNS ClusterIP |
| Root disk slightly smaller than requested | Filesystem-mode PVCs lose a few percent to filesystem overhead; request a larger disk or use a Block-mode StorageClass |