Confidential AI ･ Pod-as-CVM vs Node-as-CVM

The two confidential trust boundaries c8s supports — each pod as its own confidential VM, or the whole node as one — with the tradeoffs, the install flags, and an explicit guide to choosing.

This is the most consequential architectural decision in c8s: what is the unit of trust and attestation? There are two answers, and they correspond to two different ways of placing the Trusted Execution Environment (TEE) boundary. The layered picture (L0 = host/hypervisor, L1 = the node, L2 = the pods) makes the difference precise.

You pick the shape at install time with --cvm-mode, which is required and has no default. Switching in place is disruptive — the two shapes move security components between host and guest — so choose before you install.

Node-as-CVM (node-as-guest)

L1 (the node)  = confidential VM (SEV-SNP or TDX) of L0   ← the attested unit
L2 (the pods)  = ordinary containers in L1               ← protected by L1's encryption boundary

The entire Kubernetes node is one confidential VM (CVM) — an AMD SEV-SNP or Intel TDX enclave. L0 (the cloud or bare-metal operator) sees only ciphertext of the node. Workloads run as ordinary containers inside that node; from a pod's point of view it is a normal node that happens to be a confidential VM. A verifier checks the node's launch digest — if it matches the published image, every container on the node is running in a verified state by definition.

The control plane (kubelet, etcd, secrets) lives inside the verified node VM.

Three install shapes land here: --cvm-mode=node for confidential VMs you run yourself (the evidence comes from /dev/sev-guest or /dev/tdx-guest), --cvm-mode=gke for Google's managed confidential VMs, and --cvm-mode=aks for Azure, where the evidence rides the vTPM at /dev/tpm0 instead of a guest device. Pass --measurements <HEX_DIGEST> — the node image's launch digest, which you know before the cluster runs — and the install pins it into both internal trust boundaries at once.

Pod-as-CVM (node-as-host)

L1 (the node)  = not trusted (a plain launchpad)
L2 (each pod)  = its own per-pod confidential VM    ← the attested unit

Each pod is its own confidential enclave, via Kata and --cvm-mode=pod. The node is just the launchpad that asks KVM to start each pod's VM. A verifier checks the pod's launch digest, not the node's. Even a malicious node cannot tamper with what a pod runs — it can break the pod, but the break is detected by attestation, not silently exploited.

Both CPU TEEs ship: --hardware-platform=sev-snp (the default) renders kata-qemu-snp and --hardware-platform=tdx renders kata-qemu-tdx. A cluster picks exactly one — mixed SNP + TDX clusters are not supported.

Side-by-side

	Node-as-CVM	Pod-as-CVM
Install flag	`--cvm-mode=node` / `gke` / `aks`	`--cvm-mode=pod`
Attested unit	the whole node	each pod
Trust required in the node operator	implicit — the in-node control plane is inside the boundary	none — the host is adversarial
Isolation between tenants on a node	container-level only	per-pod hardware memory encryption
Per-pod attestation to an external verifier	no (all pods share the node digest)	yes
Measurement pinning	`--measurements` at install pins the node digest into `cds.measurements` and `ratlsMesh.measurements`	`--measurements` is refused; predict the guest digest and set those values in a values file
Image admission	baked into the node image (`node`), or the host `nri-image-policy` DaemonSet (`gke` / `aks`)	in-guest `policy-monitor`
Density / overhead	high density, one VM per node	one VM per pod; boot cost per pod
Startup latency	normal pod start	includes VM boot
Control plane in the boundary	yes (kubelet, etcd, secrets)	no — per-pod
GPU	drivers baked into the node image, measured into the node digest	NVIDIA GPU passed through per pod over VFIO
Image / build	one node image — `ghcr.io/confidential-dot-ai/c8s-base` (`:rke2`), rebuilt by c8s CI on every merge	sealed guest rootfs (`kata-guest-base`) plus a hypervisor-capable node OS (`rke2-kata`)
Works on Azure AKS	yes — SEV-SNP or Intel TDX, both via the vTPM	no (needs nesting AKS lacks)

Image admission moves with the shape

Every shape enforces the container-image allowlist, but the enforcement point differs — and the chart will not let you end up with none:

Pod-as-CVM (--cvm-mode=pod) enforces in-guest with policy-monitor, fed by an allowlist seed baked into the launch measurement plus CDS's served /allowlist. The chart fails the render if the host-side nri-image-policy, ratls-mesh, or attestation-api are left enabled alongside it: the host copies would be a second, unattested enforcement path.
--cvm-mode=node bakes a fail-closed nri-image-policy into the node image, so the chart's own copy is disabled to avoid duplicating it.
Every other cluster — gke, aks, or any non-Kata install — has the host nri-image-policy DaemonSet as its only image-admission gate. The chart therefore refuses to render with nriImagePolicy.enabled=false there; disabling it would run confidential workloads with no attested allowlist at all.

When to choose which

Choose Node-as-CVM when…

The trust unit is 'this whole node and everything on it'; you don't need per-pod isolation from the cluster operator; you want the simplest model and maximum density; the control plane should be inside the boundary. Works on any L0 that exposes a confidential VM (AMD SEV-SNP or Intel TDX) — including Azure AKS.

Choose Pod-as-CVM when…

The trust unit is each individual pod; you need per-pod isolation from the cluster operator (multi-tenant, or the tenant doesn't trust the platform); each pod must prove its exact state to an external verifier independently. Requires a bare-metal-class confidential host.

In short:

Node-as-CVM is the all-or-nothing model: verify the node once, and everything on it is trusted. Simplest, densest, and the only option on managed services without nested virtualization. The tradeoff is that you must trust the in-node control plane and you get no per-pod attestation.
Pod-as-CVM is the mutual-distrust model: the platform operator and the tenant workloads do not trust each other, and each pod carries its own attested identity. The tradeoff is one VM per pod (more overhead, slower starts) and a more complex install (Kata runtime, per-pod measurement pinning, admission enforcement).

Pod-as-CVM is not available on Azure.

Azure's hypervisor does not expose nested virtualization, so a per-pod VM cannot be launched inside an AKS node. On Azure you use Node-as-CVM, where the AKS node is a single confidential CVM and all its pods share that one boundary. Both CPU TEEs are supported there — --hardware-platform=sev-snp and --hardware-platform=tdx both attest through the Azure vTPM, since AKS exposes no guest TEE device. See Provisioning on Azure for the details and what you still get.

Implementation note

The two shapes place different demands on the node image.

Node-as-CVM works on any confidential VM image. If you supply your own, c8s CI publishes one built for it — ghcr.io/confidential-dot-ai/c8s-base (hardened kernel plus kubelet/containerd/CNI, dm-verity with IGVM/UKI), which also bakes in the attestation and image-policy components so they fall under the node's launch digest. On a managed platform that does not let you change the node image — AKS and GKE confidential node pools — you run the provider's image and the chart deploys those components as DaemonSets instead. Either way the node carries a launch digest; what differs is who builds the thing being measured, and therefore how much of the stack the digest covers.

Pod-as-CVM does constrain the node OS: it needs a hypervisor-capable one (rke2-kata, a full kernel with KVM and VFIO) whose job is to launch the sealed kata-guest-base per-pod VMs. Here the per-pod VM is the confidential unit and carries the digest that matters, so the node OS itself is outside the boundary.

The host requirements differ too. Booting a node image via IGVM needs QEMU 10.1 or newer built with --enable-igvm, which most distributions do not ship — check with qemu-system-x86_64 -object igvm-cfg,help. Pod-as-CVM needs no host QEMU at all: kata-deploy brings its own TEE-capable build.

Once you've chosen a shape, Kata Containers covers how the Pod-as-CVM path is actually built — the measured boot, the sealed guest image, the admission rules, and the guest-pull model.