Skip to content

V0.3 OPERATIONS

Fleet v0.3 operations safety

This change set adds four operator-facing foundations that are useful before Fleet expands into more runtime adapters.

1. Site maintenance / cordon

A site can be marked maintenance: true with an optional reason. Maintenance is a rollout cordon, not a shutdown command: the agent keeps reconciling the last accepted desired revision, but new rollout plans exclude the site by default.

fleetctl site-maintenance site_abc on "generator service"
fleetctl site-maintenance site_abc off

Rollout planning and creation accept includeMaintenance: true for an explicit break-glass override. Pre-flight plans return maintenanceExcluded so operators can see exactly what was filtered.

2. Scoped API tokens

Admins can create bearer credentials for CI, GitOps and automation without sharing a human password or browser session.

Token format:

zf_api_<random>

Only the SHA-256 digest is persisted. The plaintext token is returned once at creation.

Scopes:

Scope Access
read GET/HEAD/OPTIONS API calls
sites:write site metadata and site command mutations
rollouts:write revisions, groups and rollout mutations
admin all API operations

A role ceiling still applies. A viewer token may only have read; an operator token cannot receive admin.

fleetctl api-token-create github-ci operator read,rollouts:write
export ZYVOR_FLEET_API_TOKEN='zf_api_...'
fleetctl revisions

API-token requests do not require the browser CSRF marker because they are Authorization-header credentials rather than cookies.

3. Signed outgoing webhooks

Admins can register HTTP(S) webhooks with optional exact or prefix event filters such as:

{
  "name": "operations",
  "url": "https://ops.example.net/fleet/events",
  "eventKinds": ["rollout.*", "site.*"]
}

If no signing secret is supplied, Fleet generates one and returns it once. Deliveries use a structured CloudEvents-style envelope and include:

Content-Type: application/cloudevents+json
X-Zyvor-Fleet-Event: rollout.started
X-Zyvor-Fleet-Signature: sha256=<hex-hmac>

The signature is HMAC-SHA256 over the exact HTTP request body. Each webhook maintains a durable event cursor, last-delivery timestamp, last error and consecutive-failure count. Delivery is at-least-once; consumers should de-duplicate by event ID.

4. Mutation audit trail

Successful and failed operator/API mutation requests are recorded without request bodies or credentials. Agent heartbeat traffic and login/logout are excluded to avoid noise and credential-sensitive data.

Each audit row records:

  • actor (human email or API-token name)
  • HTTP method and path
  • response status
  • remote IP
  • timestamp

The log is bounded to the most recent 5,000 records and is readable by admins at GET /api/v1/audit.

Security notes

  • API token plaintext is never persisted.
  • Webhook signing secrets must be persisted because Fleet needs them to produce HMAC signatures; the existing Fleet state file remains the protection boundary and should stay mode 0600 on hosts and protected by the control-plane PVC policy on Kubernetes.
  • Webhook targets are administrator-configured and may intentionally point to private networks. Deployments that require egress restrictions should enforce them with Kubernetes NetworkPolicy/firewall policy.
  • Maintenance mode does not interrupt an already-active rollout; it prevents selection by later rollout plans. Abort/pause an active rollout separately when needed.