Per-node enrollment
Until now every node held the fleet-admin token: one leaked node meant a compromised fleet, and a node's identity was the name it chose for itself. With enrollment a node joins with a join token and receives its own credential, which can only register that node.
operator ── POST /fleet/enroll/tokens ──► fvjt_<id>.<secret> (shown once; central keeps only a hash)
node ── POST /fleet/enroll (Bearer fvjt_…) {name, node_id} ──► fvnc_<node_id>.<generation>.<mac>
node ── POST /fleet/register (Bearer fvnc_…) ──► heartbeats, as before
Rules
- Join tokens can expire (
ttl_secs), be limited tomax_usesnodes and to aname_pattern(rack1-*), and be revoked (DELETE /fleet/enroll/tokens/{id}; nodes already enrolled keep their credentials). A use is counted the first time a node enrolls. 150 concurrent enrolls on a token withmax_uses100 yield exactly 100 credentials. node_idis generated by the node once (<state-dir>/node-id, mode 0600). Enrolling the same name with the samenode_idand token again returns the same credential and costs no use, so a lost response does not burn one. Anothernode_idfor an enrolled name is409and leaves the original intact; so is onenode_idunder two names. To move a name to new hardware the operator runsPOST /fleet/nodes/{name}/rebind, which kills the old credential.- Credentials are not stored: the mac is an HMAC-SHA256 of name, id and generation under a secret only central holds
(
fleet-enroll.json, mode 0600). A credential works forPOST /fleet/registeras that node (another name is403),POST /fleet/enroll/renew, andGET /fleet/nodes/{self}/overlays. Everything else is403. It is neither an operator nor a tenant token, and a join token is good for/fleet/enrollonly. - Revoke (
POST /fleet/nodes/{name}/revoke) stops a node that is still heartbeating, unlikederegister, which refuses a healthy node. Rotate (POST /fleet/nodes/{name}/rotate, or the node's own daily renew) issues the next generation; the previous one keeps working for an hour. Deregistering a node drops its credential. - Node side:
fluxvm-agent node --token-file <file with fvjt_…> --state-dir …exchanges the token once, stores the credential 0600 and uses it from then on, also when restarted without the join token. It refuses anode-idornode-credentialfile that group or others can read. Pass the join token through a file or systemdLoadCredential=, never argv; never bake it into an image. - Enrollment is rate limited separately from heartbeats (500/s, burst 1000, 256 in flight; a refusal is
429withRetry-After, which nodes andsim-fleethonour).
Migration: --node-auth
| Mode | Fleet token registers nodes | Node credentials | Use |
|---|---|---|---|
legacy | yes | refused, /fleet/enroll is 404 | enrollment off |
both (default) | yes | yes | while nodes migrate |
enrolled | no (403 on /fleet/register) | yes | after every node has enrolled |
Check progress with GET /fleet/enroll/nodes against GET /fleet/nodes. The plan flips the default to enrolled one
release after migration tooling ships; that is a separate, deliberate change.
What was tested
cargo test -p fluxvm-agent covers the rules above (including the 150/100 race and an HTTP walk through every allowed and
refused route, and a restart of central) and the node's file permission checks. scripts/test-enroll-invariants.sh runs a
real central and sim-fleet: exactly 100 of 150 concurrent enrolls, a mixed enrolled/legacy fleet of 400 registering
exactly once, the registry and the enrollments surviving kill -9, and enrolled mode refusing the fleet token. A real
fluxvm-agent node enrolled with a join token and kept working when restarted without it.
Not tested / needs your review before relying on it: this is authentication code and has had no external security review; TLS between nodes and central is whatever you configure (credentials travel in a bearer header); there is no per-IP throttle on failed join-token guesses (the secrets are 244 random bits, and attempts are rate limited globally); mTLS node identity is deferred as decided; real-fleet behaviour (10,000 distinct clients) is untested.