Create an encrypted volume

Build the image and store the key

Prepare ./llama-3.1-8b as the plaintext source directory, readable by the workload's numeric runAsUser after packaging. Use an empty ./volume-build directory on protected storage for intermediates. The following command creates ./weights.img and ./weights.escrow.json, then stores the complete key blob at /tenant-a/volumes/weights:

c8s volume create \
  --name weights \
  --source ./llama-3.1-8b \
  --out ./weights.img \
  --path /tenant-a/volumes/weights \
  --escrow-out ./weights.escrow.json \
  --work-dir ./volume-build \
  --node node-1 \
  --url "https://<CDS_HOST>:<CDS_PORT>" \
  --image-policy-file ./cds-pin.json \
  --mesh-ca ./mesh-ca.pem \
  --operator-key ./operator.key

The image and escrow output paths must not already exist. The default format is erofs with an encrypted dm-verity tree; its size follows the source contents. Success prints the immutable data-block count or mutable image size, stored key path, escrow path, device serial, pod annotations, node selector, and policy fragments. The mount rules are covered in Grant the workload the key path.

--jobs defaults to 0, selecting min(GOMAXPROCS, 16) encryption workers. A positive value sets the worker count; negative values are refused. The worker count does not change the image format. Cleanup attempts to delete build intermediates on return; deletion failures can leave plaintext behind. File deletion does not securely erase that data. A failed build can also leave an incomplete output image.

For a mutable volume, prepare ./scratch-source, even if empty, with the workload's numeric runAsUser as its owner. Its contents are preloaded, and its owner becomes the ext4 root directory's owner. This command creates a 50 GiB writable image and a separate key path:

c8s volume create --mutable \
  --name scratch \
  --source ./scratch-source \
  --size 50Gi \
  --out ./scratch.img \
  --path /tenant-a/volumes/scratch \
  --escrow-out ./scratch.escrow.json \
  --work-dir ./volume-build \
  --node node-1 \
  --url "https://<CDS_HOST>:<CDS_PORT>" \
  --image-policy-file ./cds-pin.json \
  --mesh-ca ./mesh-ca.pem \
  --operator-key ./operator.key

Without --source, a mutable volume starts empty with a root-owned filesystem root. A non-root pod cannot create files there. With --source and no --size, c8s infers and prints the size. --size accepts bytes or quantities such as 50Gi, 500M, and 2Ti; the minimum is 16 MiB. Immutable volumes refuse --size.

The CDS write is create-only. An occupied key path is refused. A new build generates a new key, so use a new path for it. After a storage or connection failure, inspect the command result and preserve any completed image and escrow. If the image and escrow exist but the CDS write failed, store that escrow with the next step's command; rerunning volume create generates another key. --dry-run still builds the image and writes escrow, but skips CDS.

Keep the escrow file

Retain weights.escrow.json on durable, access-controlled storage. It is the complete c8s.volume/v1 JSON blob: a base64-encoded 64-byte XTS key plus immutable verity parameters, or mutable: true for a writable volume. c8s creates the escrow file with mode 0600. Keep its association with the ciphertext and CDS path.

A singleton CDS restart loses its memory-only secret store. An HA replica can recover from surviving replicas; losing all volatile cluster state requires escrow restoration for future key releases. Existing open mounts are not automatically closed by CDS state loss.

To restore the original key to an empty path, or finish a build whose CDS write failed, run:

c8s secrets put /tenant-a/volumes/weights \
  --from-file ./weights.escrow.json \
  --url "https://<CDS_HOST>:<CDS_PORT>" \
  --image-policy-file ./cds-pin.json \
  --operator-key ./operator.key \
  --mesh-ca ./mesh-ca.pem

This stores the file's bytes unchanged. For scratch, substitute its path and escrow file. An occupied path is refused unless you explicitly use --overwrite; replacing a key blob can strand ciphertext that needs the existing value. Restore the original blob, not a newly generated key.

For HTTPS writes, --mesh-ca requires every served CA certificate to be present in the pinned bundle, comparing encoded certificates. It supplements the CDS image/configuration pin. --force skips only this additional CA check. If the CA changes, retain the old anchor while you investigate. To fetch a candidate into a separate file:

c8s cds ca \
  --url "https://<CDS_HOST>:<CDS_PORT>" \
  --image-policy-file ./cds-pin.json \
  --out ./mesh-ca.candidate.pem

The command writes the currently served bundle to mesh-ca.candidate.pem. Fetching it does not establish continuity with your previous CDS. Follow Obtain the mesh CA through the pinned CDS before accepting it as mesh-ca.pem.

Present the image to a node

Transfer the ciphertext image to storage your VM environment can expose as a raw block device. Keep plaintext and escrow on the trusted provisioning side. The node selects the device by serial c8s-vol-weights; use c8s-vol-scratch for the mutable example.

For QEMU/KVM, adapt this launch fragment to the VM definition. <IMAGE> is the absolute path to the ciphertext on the hypervisor host; applying the fragment adds a disk to the VM:

-drive if=none,id=volweights,file=<IMAGE>,format=raw -device virtio-blk-pci,drive=volweights,serial=c8s-vol-weights

For hot attachment, provision a virtio-scsi controller at VM launch and use the hypervisor's supported SCSI attachment procedure. The baked node image disables PCI hotplug. volumed reads virtio serials from sysfs and SCSI serials from VPD page 0x80.

On a Linux confidential node with configfs mounted and LIO kernel support, the optional attach helper can expose a local ciphertext file as a SCSI device. It requires root and changes the node's kernel target configuration. Copy the completed image to /var/lib/c8s/weights.img on that node before running:

sudo c8s volume attach weights --image /var/lib/c8s/weights.img

Success reports serial c8s-vol-weights. This helper is unsupported on the baked c8s node image, which omits LIO and disables module loading after boot; present its disk through the hypervisor instead.

The serial selects storage; it does not authenticate it. Duplicate matching serials are refused. Immutable reads must verify against the trusted root hash. A mutable mount can fail on unrelated ciphertext, but successful mounting does not prove its integrity or identity.

Schedule the pod on the device's node using kubernetes.io/hostname: node-1. The build's --node option only prints this selector; it does not attach storage or schedule anything.

Request the volume from the pod

Before deriving policy, merge these fields into deploy.json, retaining the rest of your Deployment. Replace <NAMESPACE> with the injection-enabled namespace chosen above:

{
  "metadata": {"namespace": "<NAMESPACE>"},
  "spec": {
    "template": {
      "metadata": {
        "annotations": {
          "confidential.ai/cw": "llama-infer",
          "confidential.ai/c8s-volumes": "weights=/tenant-a/volumes/weights"
        }
      },
      "spec": {
        "nodeSelector": {"kubernetes.io/hostname": "node-1"}
      }
    }
  }
}

For a mutable volume, substitute scratch=/tenant-a/volumes/scratch and merge these fields into the Deployment's top-level spec now:

{
  "spec": {
    "replicas": 1,
    "strategy": {"type": "Recreate"}
  }
}

See Run one writer at a time for the storage-sharing boundary. Each annotation entry is NAME=/store/path, comma-separated. Names are DNS-1123 labels of at most 12 characters. For weights, the application path is /mnt/c8s-data/c8s-volumes/weights. To move the base directory, set confidential.ai/c8s-volume-dir at or below /mnt/c8s-data and change the application paths and exact mount rules accordingly.

The webhook injects the c8s-volume native sidecar after c8s-cert-wait and mounts a default-medium emptyDir named c8s-volume-weights into every application container with HostToContainer propagation. volumed mounts the opened filesystem over that directory: read-only for immutable, read-write for mutable. File ownership comes from the built image; the daemon does not change it at mount time.

Leave the reserved c8s-volume container name to the injector. A predeclared volume under c8s-volume- must be a default-medium emptyDir; a PVC or hostPath there is rejected. See Workload annotations.

Consumers must wait for each opened filesystem before using it. The placeholder is already a mount, so mountpoint -q alone cannot distinguish it. Its backing filesystem can also be ext4. Application init containers must not block launches needed for the release decision by waiting on this volume.

For a Linux application with Python 3 and readable /proc/self/mountinfo, this example checks the default volume path's device, filesystem type, and c8s mapper source. Integrate wait_for_volume into the consuming application's startup code, in the same process that uses the volume. The bottom block demonstrates one checked read; adapt it to the application's actual data initialization. Build that integration into the application image before pinning its digest in deploy.json. Replace <FILE_NAME> with the name of a readable regular file directly inside the immutable source directory. Use a single filename without /, other than . or ..; the example refuses a symlink. The read prints only a non-secret result:

import os
import re
import time


def wait_for_volume(name, mutable=False):
    target = f"/mnt/c8s-data/c8s-volumes/{name}"
    kind, fs_type = ("crypt", "ext4") if mutable else ("verity", "erofs")
    source = re.compile(rf"/dev/mapper/c8s-{kind}-[0-9a-f-]+-{re.escape(name)}")
    while True:
        root = os.open(target, os.O_RDONLY | os.O_DIRECTORY | os.O_CLOEXEC)
        try:
            dev = os.fstat(root).st_dev
            device = f"{os.major(dev)}:{os.minor(dev)}"
            with open("/proc/self/mountinfo") as mounts:
                for line in mounts:
                    before, after = line.split(" - ", 1)
                    left, right = before.split(), after.split()
                    if (left[2:5] == [device, "/", target]
                            and right[0] == fs_type
                            and source.fullmatch(right[1])):
                        return root
        except BaseException:
            os.close(root)
            raise
        os.close(root)
        time.sleep(1)


root = wait_for_volume("weights")
try:
    data = os.open("<FILE_NAME>", os.O_RDONLY | os.O_NOFOLLOW, dir_fd=root)
    with os.fdopen(data, "rb") as stream:
        stream.read(1)
    print("weights mount ready", flush=True)
finally:
    os.close(root)

The open directory handle binds the check and subsequent read to the same filesystem. Keep the returned handle open for the application's volume I/O, use dir_fd=root for that I/O, and close it only when that work is finished. For mutable I/O, use wait_for_volume("scratch", mutable=True) before creating or opening files relative to the returned handle. Other application runtimes need the equivalent directory-handle and mount checks. Validate mount propagation and the reported mapper source in your runtime; this example is a source-derived readiness check, not a runtime-tested deployment recipe.

Make the application's readiness signal become ready only after the mount check and its required mounted-data initialization succeed. Configure the Deployment's readiness probe to use that signal; printing the check result does not set Kubernetes readiness.

Finalize the image digest, launch command, arguments, environment, annotations, namespace, readiness probe, and scheduling in deploy.json before the next step.

Grant the workload the key path

Add a read grant to the workload entry in the allowlist. Use the exact key path: a grant for /tenant-a/volumes/** permits release of every key beneath it. A read grant releases the whole blob, including write capability for a mutable volume; file writes do not require a secret store write grant.

Every application init and main container must pin its command, arguments, and mounts. A matching digest alone is insufficient. The volume is a data mount at /mnt/c8s-data/c8s-volumes/weights. For a single application container, save this JSON as mounts.json:

{
  "<APP_CONTAINER>": {
    "policy": "exact",
    "rules": [
      {"destination": "/mnt/c8s-data/c8s-volumes/weights", "kind": "data"}
    ]
  }
}

Replace <APP_CONTAINER> with the container name in deploy.json. Include an entry for every application init and main container. exact compares the complete non-platform mount set: add one data rule for each further volume and the other mounts the container uses. An application requesting secrets also needs {"destination": "/run/c8s/secrets", "kind": "emptyDir"}. Private mesh credentials stay in platform containers, so application mount rules do not include /etc/c8s/certs.

Arbitrary host and GPU socket mounts are ineligible for grants. A proven read-only nodeRootFile rule can qualify; see Mount policy.

For the mutable example, change the mount destination to /mnt/c8s-data/c8s-volumes/scratch and the read grant below to /tenant-a/volumes/scratch. Derive entry.json from the finalized manifest and mount map:

c8s allowlist derive llama-infer deploy.json \
  --env any \
  --secret-read /tenant-a/volumes/weights \
  --mounts-file mounts.json > entry.json

The redirection creates or replaces entry.json with a name-keyed workload entry map. --env any leaves environment values unconstrained; use --env-file when you have the complete intended launch environment. Review the derived image digests, commands, arguments, mounts, and grant. When replacing an existing entry, retain its other intended grants and egress policy: apply replaces that entry in full.

c8s allowlist lint entry.json
c8s allowlist apply entry.json \
  --url "https://<CDS_HOST>:<CDS_PORT>" \
  --image-policy-file ./cds-pin.json \
  --operator-key ./operator.key
c8s allowlist get llama-infer \
  --url "https://<CDS_HOST>:<CDS_PORT>" \
  --image-policy-file ./cds-pin.json

Resolve lint errors and review warnings before applying. apply publishes a complete signed policy document while preserving other workload entries. In the returned entry, check secrets.read for /tenant-a/volumes/weights and each container's exact mount policy. Follow Manage the allowlist to check node propagation before deployment.

CDS rechecks the current policy at release against a uniquely matching sandbox admission record. That record includes admitted launches after containers exit; it does not establish application readiness. volumed validates the blob and pod mount target but does not repeat the CDS grant decision.

Deploy the prepared workload

After publishing the matching policy, apply the finalized manifest with the authorized context and namespace:

kubectl --context "<KUBE_CONTEXT>" --namespace "<NAMESPACE>" apply -f deploy.json

This creates or updates the Kubernetes workload. Writes through the placeholder do not reach the encrypted device.

Verify the mount landed

Read the fetcher's logs and your application's ordinary startup check:

kubectl --context "<KUBE_CONTEXT>" --namespace "<NAMESPACE>" logs "<POD>" -c c8s-volume
kubectl --context "<KUBE_CONTEXT>" --namespace "<NAMESPACE>" logs "<POD>" -c "<APP_CONTAINER>"

Use the same context and namespace as deployment. Replace <POD> and <APP_CONTAINER> with the workload's current pod name and application container. The fetcher logs volumes opened after all requested opens succeed. The Python startup check above logs weights mount ready after checking the filesystem and reading from the retained directory handle. A successful immutable mount does not scan every block; verity checks contents as the application reads them.

For a mutable volume, make the application's test path wait for the ext4 mount, write and flush a non-secret marker, and report its identifier. Recreate the test pod on the same node and backing device, then have the application read and report that identifier. This checks persistence across that restart; it does not establish rollback protection. Use ordinary application code or an authenticated application endpoint for these checks. Locked baked nodes refuse arbitrary runtime exec.

Startup validation or pin-resolution errors exit the fetcher before its retry loop. Once that loop starts, release and open failures retry indefinitely with the defaults, including persistent failures such as a malformed key blob. The first delay is five seconds, doubling to a thirty-second cap. The native sidecar can restart an exited process.

A 404 from the key read can mean either an absent value or a denied path. Check the served grant and fetcher logs, then correct policy or restore the original escrow as appropriate. See Troubleshoot for release failures.

Optional sandbox diagnosis. Run secrets explain only when the trusted deployment operator supplies the affected pod's runtime sandbox ID from node runtime/inventory records or captured evidence. It is a separate input, not the Kubernetes pod UID. With that input:

c8s secrets explain --sandbox "<SANDBOX_ID>" \
  --url "https://<CDS_HOST>:<CDS_PORT>" \
  --image-policy-file ./cds-pin.json \
  --operator-key ./operator.key

Replace <SANDBOX_ID> with the supplied runtime identifier. An operator can also supply an evidence-bearing public workload certificate or a supported workload evidence endpoint; Verify a workload describes checking its sandbox stamp. That guide does not retrieve the identifier from an ordinary application port. Application containers cannot read the platform's private credentials. If these operator-supplied inputs are unavailable, use the logs and policy readback above.

After success, the sidecar idles until termination. It does not watch for replacement key blobs or grant removal, and those changes do not close an existing mount.