Create an encrypted volume

Package a directory into an encrypted c8s volume with c8s volume create, keep the escrow file, attach the image to a node as a raw block device, grant the workload its key path, and mount it into a pod.

An encrypted volume is data too large to be a secret — model weights are the case it is built for. It sits as ciphertext on storage the untrusted host reads freely and decrypts only inside a TEE, for a workload the allowlist names.

This page is the procedure. For the artifact's layers, what happens at mount time, and what the design does and does not defend against, see Encrypted volumes.

volumed is off by default (volumed.enabled: false) and nothing here works without it. Turn it on at install with a values file — see volumed, the node agent.

Volumes also require Node-as-CVM; the webhook rejects confidential.ai/c8s-volumes at admission under Pod-as-CVM.

Build the image and store the key

c8s volume create formats the filesystem, builds the hash tree, generates a key, encrypts, and PUTs the blob to the CDS secret store. It modifies no workload:

c8s volume create \
  --name weights \
  --source ./llama-3.1-8b \
  --out ./weights.img \
  --path /tenant-a/volumes/weights \
  --escrow-out ./weights.escrow.json \
  --node node-1 \
  --url https://cds.example \
  --measurements-file ./measurements.txt \
  --operator-key ./operator.key
FlagRequiredEffect
--nameyesVolume name. A DNS-1123 label of at most 12 characters — it forms the device serial c8s-vol-<name> and the directory the plaintext appears in. A longer name is rejected.
--sourceyesDirectory whose contents become the volume.
--outyesWhere to write the encrypted image. Must not already exist.
--pathyesSecret-store path for the key, e.g. /tenant-a/volumes/weights. Absolute, clean, no wildcards.
--escrow-outyesWhere to write the key blob you must keep. Written 0600; refuses to overwrite.
--nodenoNode holding the device. Emitted as a nodeSelector in the printed output.
--work-dirnoDirectory for build intermediates. Default: a temp dir. The intermediates — the plaintext image and the tree — are removed either way, success or failure.
--dry-runnoBuild the image and write escrow, but do not call CDS.

It also takes the shared CDS connection and credential flags: --url, --measurements / --measurements-file, --operator-key (or C8S_OPERATOR_KEY), --timeout (default 15s), and --insecure. The write is authorized by an operator EC key whose public half CDS pins via c8s install --operator-keys.

The build needs mkfs.erofs and veritysetup on the machine running it. It does not need root, a loop device, or cryptsetup — the encryption is done in process.

The key is generated per volume and never taken from you. There is no flag to supply one.

Output:

+ ./weights.img (26214400 data blocks)
+ key stored at /tenant-a/volumes/weights
+ key escrowed to ./weights.escrow.json — keep it; a CDS restart needs it

Attach ./weights.img to the node as a raw block device with serial c8s-vol-weights.

Pod annotations:
  confidential.ai/cw: <workload-id>
  confidential.ai/c8s-volumes: "weights=/tenant-a/volumes/weights"

Pod nodeSelector (the device is on one node):
  kubernetes.io/hostname: node-1

Allowlist grant for the workload entry (read-only, exact path):
  "secrets": {"policy": "allow", "read": ["/tenant-a/volumes/weights"]}

A subtree grant would cover every volume beneath it, so this names one path.

The store write is create-only. A path that already holds a value is refused: a volume's key and its ciphertext are one unit, so replacing the key at a path some volume already uses strands that volume rather than rotating anything. Choose another path.

Keep the escrow file

CDS keeps secrets in process memory and nowhere else. A CDS restart makes every volume in the cluster unopenable until its key is written back, and the escrow file is what you write it back from:

c8s secrets put /tenant-a/volumes/weights \
  --from-file ./weights.escrow.json \
  --url https://cds.example \
  --operator-key ./operator.key

The escrow file is the only copy of the key outside the CDS process.

Lose it and restart CDS, and the ciphertext is unrecoverable — there is no other copy, no versioning, and no recovery path. Its compromise is equivalent to handing over the plaintext, permanently. Store escrow files somewhere durable and access-controlled.

Attach the image to a node

The image is ciphertext. Copy it to the node by any means, including through the untrusted host — that the host holds the bytes is the premise, not a compromise of it.

Attach it as a raw block device whose virtio serial is c8s-vol-<name>. volumed finds it by reading serial under /sys/block, so no udev rules are needed. A confidential node has no persistent writable storage — the root overlay is reformatted on every boot — so a volume has to be its own device rather than a file on the node's filesystem.

The serial is a selector, not a trust input. The host chooses it and answers the query per read. Pointing a pod at the wrong device fails closed: the wrong key produces noise, and verity refuses it. Two devices claiming the same serial are refused outright rather than resolved by scan order.

Because the device lives on one node, the pod must be scheduled there. create emits the matching nodeSelector.

Grant the workload the key path

Release is gated on the workload entry's secrets grant in the allowlist:

"secrets": { "policy": "allow", "read": ["/tenant-a/volumes/weights"] }

Name the exact path, not a subtree. /tenant-a/volumes/** grants every volume beneath it, and the annotation naming which volume to open is host-written. create prints an exact-path grant for this reason.

read only. A volume is mounted read-only, so a write grant says nothing about whether a workload may see the plaintext.

Request the volume from the pod

A pod names its volumes in an annotation:

metadata:
  annotations:
    confidential.ai/cw: llama-infer
    confidential.ai/c8s-volumes: "weights=/tenant-a/volumes/weights"
    confidential.ai/c8s-volume-dir: "/models"    # optional

Each entry is NAME=/store/path, comma-separated. NAME selects the node's device by its c8s-vol-<NAME> serial and names the directory the plaintext appears in under the volume dir — above, /models/weights. Without confidential.ai/c8s-volume-dir the default is /run/c8s/volumes.

For a pod carrying confidential.ai/cw, the webhook then injects:

  • a c8s-volume native sidecar, ordered after c8s-cert-wait — it authenticates with the leaf that sidecar writes;
  • per volume, a default-medium emptyDir named c8s-volume-<NAME>, mounted into every container read-only with mountPropagation: HostToContainer.

Both names are reserved. A pod may not declare its own container called c8s-volume, and a volume it pre-declares under the c8s-volume- prefix must be a default-medium emptyDir or be omitted entirely — a hostPath, a PVC, or a memory-backed emptyDir is rejected at admission. See Reserved containers and volumes.

Verify the mount landed:

kubectl exec -n <NAMESPACE> <POD> -- ls /models/weights

Expect the volume's files. An empty directory means the mount has not landed yet — check the c8s-volume sidecar's logs.

See also