Skip to main content

atlas-native CSI driver

crates/atlas-native-csi (bin atlas-native-csi, image Dockerfile.native-csi) is a Container Storage Interface driver for atlas-native filesystems (NATIVE_FS.md). Driver name: native.atlas.zyvor.ai.

Kubernetes objectatlas-native object
PersistentVolume (from a PVC)filesystem (POST /v1/fs)
VolumeSnapshotfilesystem snapshot (POST /v1/fs/{fs}/snapshots)
PVC with a VolumeSnapshot dataSourceclone of the snapshot (POST /v1/fs-snapshots/{id}/clone)
PVC with a PVC dataSourceclone of a transient snapshot of the source, deleted once the clone exists
Pod mountone atlas-native-mount FUSE process on the pod's volume path
PVC sizethe filesystem's byte quota (PUT /v1/fs/{fs}/quota); expanding the PVC raises it

Every access mode works (ReadWriteOnce, ReadOnlyMany, ReadWriteMany, ReadWriteOncePod): each pod's mount is its own client of a shared filesystem, and cross-mount consistency is the filesystem's (NATIVE_FS.md, "Consistency"). Not supported: block volumes, shrinking a volume, inode limits from the CSI side, and topology.

Install​

The atlas-native Helm chart deploys it next to the storage nodes:

helm upgrade --install native deploy/helm/atlas-native \
  --set csi.enabled=true \
  --set csi.snapshots.enabled=true       # needs the VolumeSnapshot CRDs and snapshot-controller

This adds the CSIDriver object (attachRequired: false, fsGroupPolicy: File), a controller Deployment (atlas-native-csi with the external-provisioner, external-resizer and, with snapshots enabled, external-snapshotter sidecars, leader-elected), a node DaemonSet (atlas-native-csi plus the node-driver-registrar), the StorageClass atlas-native-fs, and the VolumeSnapshotClass atlas-native-fs. Both plugins reach the storage nodes through their per-pod DNS names with the release's API token Secret; with httpTls.enabled they trust the Secret's ca.crt (only that key is mounted). httpTls.requireClientCert is not supported with the CSI driver yet.

The node plugin runs privileged with /dev/fuse and the kubelet directory mounted Bidirectional, so the FUSE mounts it makes are visible to the kubelet and the pods.

apiVersion: v1
kind: PersistentVolumeClaim
metadata: { name: checkpoints }
spec:
  accessModes: [ReadWriteMany]
  storageClassName: atlas-native-fs
  resources: { requests: { storage: 100Gi } }

StorageClass options​

FieldEffect
parameters.extentBytes (csi.storageClass.extentBytes)Extent grid of new filesystems in bytes (default: the cluster's extent_bytes)
parameters.enforceCapacity (csi.storageClass.enforceCapacity)"true" (default): the PVC's size is the filesystem's byte quota. "false": the filesystem is thin and the size is only reported
allowVolumeExpansiontrue in the chart's StorageClass
mountOptions (csi.storageClass.mountOptions)atlas-native-mount flags: ro, acl (enforce POSIX ACLs), cache-leases, direct-reads, and <flag>=<n> for ttl-ms, writeback-bytes, writeback-parallel, readahead-bytes, max-io-bytes, fuse-threads, session-ttl-ms, retry-secs

Any other parameter or mount option is refused (InvalidArgument), so a StorageClass can't point a mount at another endpoint or credential.

Behaviour​

  • Capacity. With enforceCapacity on, CreateVolume sets the filesystem's byte quota to the requested size (required_bytes, else limit_bytes; no request, no quota). Writes past it fail with EDQUOT in the pod (NATIVE_FS.md, "Quotas", including how write-back delays the error to fsync/close), and df in the pod shows the PVC's size. A clone made from a snapshot or a PVC gets the byte limit of its own request, not the source's.

  • Expansion (ControllerExpandVolume, ONLINE): raises the byte quota to the new size and never lowers it. No node step is needed: mounted pods see the new size at once. A volume without a byte quota (enforceCapacity: "false", or created before this driver set quotas) stays thin; expansion only reports the new size, so enforcing an existing volume means setting its quota through the API.

  • Volume stats (NodeGetVolumeStats, with volume condition): bytes and inodes from statfs on the pod's mount, so kubelet's kubelet_volume_stats_* show the quota as capacity. A mount whose FUSE process is gone, or that doesn't answer within 10 s, is reported abnormal instead of failing the call.

  • Idempotent creates. The volume id is the CSI name when it is a valid atlas-native id of at most 48 characters (the provisioner's pvc-<uuid> and snapshot-<uuid>), else csi- plus 40 hex digits of its SHA-256. A retried CreateVolume or CreateSnapshot lands on the object the first attempt made; the same snapshot name on a different source volume is AlreadyExists.

  • Deletes of a missing volume or snapshot succeed. Clones keep their own references to the shared extents, so deleting a source volume or snapshot never affects its clones.

  • Publish spawns atlas-native-mount --fs <id> --allow-other [--read-only] <target> (the connection flags pass file paths only, never the token itself) and waits up to --mount-timeout-secs (30) for the target to appear in /proc/self/mountinfo. Publishing a healthy mounted target again is a no-op; a target whose FUSE process is gone (ENOTCONN) is unmounted and mounted again.

  • Unpublish unmounts with umount2(2) (falling back to a lazy unmount), reaps the mount process, and removes the target directory. It does not use fusermount3 -u: Ubuntu's AppArmor profile for fusermount3 refuses unmounts under the kubelet root even from a privileged container (found on the lab).

  • Restarting a node plugin pod breaks the mounts it made. The FUSE processes are its children, so a DaemonSet rollout or plugin crash leaves the pods on that node with Transport endpoint is not connected until they are restarted; the replacement plugin then unmounts the dead mounts and publishes fresh ones. The chart's node DaemonSet therefore uses updateStrategy: OnDelete (csi.node.updateStrategy): a chart upgrade doesn't roll it; drain a node, then delete its plugin pod to pick up a new image.

Verification​

cargo test -p atlas-native-csi: the controller over a real Unix socket against an in-process cluster (idempotent create, hashed names, block refused, snapshot create retry with a stable creation time, AlreadyExists across volumes, snapshot and volume clones checked through the FUSE operations layer, NotFound sources, validate, idempotent deletes, clones outliving their sources), the node service's validation, mount-option whitelisting, and mountinfo parsing. Capacity: the PVC size becoming the quota, expansion raising it and never lowering it, enforceCapacity: "false" staying thin, a snapshot clone getting its own size, NotFound on a missing volume, and EDQUOT on a 64 KiB volume until it is expanded. statvfs mapping and node capabilities.

Live on the lab's single-node k3s (v1.36, Ubuntu 26.04, chart with replicas=1, local-path state PVC), 2026-10-05:

  • The node plugin registered with the kubelet; a ReadWriteMany PVC bound and two pods with different uids mounted it at once (fuse mount, allow_other, default_permissions). An 8 MiB random file written by one pod read back with the same MD5 in the other, and fsGroup ownership was applied.
  • A VolumeSnapshot became readyToUse; a PVC restored from it held the files from before the snapshot and not a file written after it, while a PVC cloned from the live PVC held both, with matching checksums.
  • After deleting the node plugin pod, the existing mounts returned ENOTCONN (as described above). The first build's unpublish then failed on the fusermount3 AppArmor denial; with the umount2 fix, kubelet's retry unpublished the dead mount and a recreated pod remounted the volume.
  • Deleting the pods left no FUSE mounts or mount processes on the host; deleting the VolumeSnapshot and PVCs left GET /v1/fs and GET /v1/fs-snapshots empty.

Capacity and expansion, same lab, 2026-10-05 (StorageClass with extentBytes: 65536):

  • A 1Mi PVC: df in the pod showed 1024 KiB. 768 KiB written with dd conv=fsync succeeded; a further 512 KiB failed with Disk quota exceeded, and usage stayed at 768 KiB.
  • Patching the PVC to 4Mi: the external-resizer reported VolumeResizeSuccessful, the PVC and PV showed 4Mi, and df in the same running pod showed 4096 KiB; the 512 KiB write then succeeded.
  • The kubelet stats summary showed the volume's capacityBytes as 4194304 with its used bytes and inodes from NodeGetVolumeStats.

Not yet verified: multi-node clusters (pods on different nodes sharing an RWX volume), HTTPS endpoints, and throughput through the CSI mount (it is the same atlas-native-mount measured in NATIVE_FS.md).