Node CVM launch contract

What the c8s node image expects from a hypervisor — published artifacts per platform, supported vCPU counts, disk serials and labels, the opkeydata and joindata disk contracts, network ports, and the manifest fields c8s pins.

The c8s node image is a measured Confidential Kubernetes node: a dm-verity root carrying a single-node RKE2 server, the baked admission inventory, and the attestation-api. It is published as an OCI artifact of raw files and launched by a hypervisor outside its trust boundary.

This page is the contract between that hypervisor and the image. The procedure for a QEMU host is Boot a node CVM on bare metal.

Published artifacts

Repository: ghcr.io/confidential-dot-ai/node-guest-base. Tags are platform-qualified: rke2-snp and rke2-tdx are the raw-file artifacts, rke2-snp-cdi and rke2-tdx-cdi the KubeVirt disk-image form of the same build. Every release tag names both the platform and the version, as in rke2-tdx-v0.1.0, and each one is exact.

FileAMD SEV-SNPIntel TDXContents
disk.rawyesyesdm-verity root filesystem, mounted read-only
uki.efiyesyesunified kernel image (kernel, initrd, cmdline)
combined-initrd.imgyesyesinitrd as a standalone file
OVMF.fdyesfirmware measured into the IGVM
OVMF.tdx.fdyesTDVF firmware, covered by MRTD
guest-smp<N>.igvmyesone measured launch image per vCPU count
dsdt.amlyesyesACPI differentiated system description table
roothashyesyesdm-verity root hash of disk.raw, as bare hex
manifest.jsonyesyesbuild inputs and the image's measured identity

The two platforms establish identity differently. SEV-SNP measures firmware and kernel together as one IGVM launch digest. Intel TDX measures the TDVF firmware as MRTD and records the guest kernel and root filesystem in RTMR[1] and RTMR[2].

Supported vCPU counts

An SEV-SNP launch measurement covers initial vCPU state, so one image yields one launch digest per vCPU count and the build ships one measured IGVM for each.

-smpIGVM fileLaunch digest
2guest-smp2.igvmsnp_variants[smp=2].measurement.snp_launch_digest
4guest-smp4.igvmsnp_variants[smp=4].measurement.snp_launch_digest
8guest-smp8.igvmsnp_variants[smp=8].measurement.snp_launch_digest
16guest-smp16.igvmsnp_variants[smp=16].measurement.snp_launch_digest

Each launch uses the IGVM matching its -smp value. An IGVM launched at another count produces a report whose measurement is not the pinned one: the node boots normally and is then refused credentials.

On Intel TDX the identity does not vary with vCPU count: the pinned tuple is mrtd, rtmr1, and rtmr2. Guest RAM is in neither platform's launch identity; the build default is 16 GB.

Disks

PurposeHow the guest finds itRequiredAbsent
rootfirst virtio-blk device, attached read-onlyyesno boot
scratch overlaydevice serial confai-scratch, at least 64 GByesfalls back to a 2 GB tmpfs; RKE2 wedges when it fills
containerd image cachedevice serial confai-containerd, or filesystem label containerdnofalls back to a 32 GB tmpfs
model weightsdevice serial confai-modelsnothe mount is skipped
operator keyISO9660 volume label opkeydatafor credential releasethe credential-release unit is skipped
cluster joinISO9660 volume label joindatafor multi-nodethe node is a single-node server
encrypted volumedevice serial c8s-vol-<name>per volumethe volume cannot be opened

Notes on discovery:

  • A disk serial is set on the -device. QEMU rejects serial= on a raw -drive with Block format 'raw' does not support the option 'serial'.
  • The scratch and cache disks are handed over blank. The guest formats and encrypts each per boot with a key that never leaves the CVM, so nothing on them survives a relaunch.
  • The weights disk is a pre-populated ext4 filesystem, mounted read-only at /var/lib/models with nodev,nosuid,noexec. Its contents stay readable by the host, and integrity rests on the workload checking model digests.
  • Volume names are DNS-1123 labels of at most 12 characters, because the serial holds c8s-vol- plus the name. See Create an encrypted volume.
  • SCSI transports surface a serial only under /dev/disk/by-id/, virtio-blk only in sysfs. The guest scans both.

The opkeydata disk

An ISO9660 filesystem whose volume label is opkeydata, carrying one file:

FileContents
pubkeythe operator's ECDSA public key, PKIX PEM, as written by openssl ec -pubout

The bytes are hashed verbatim, armor and trailing newline included; any re-encoding changes the binding and verification fails. Rock Ridge extensions are required for the lowercase filename.

PlatformBinding
AMD SEV-SNPthe launcher commits SHA-256(pubkey) as the report's HOSTDATA at launch
Intel TDXthe measured initrd extends SHA-384(pubkey) into RTMR[3] before switch_root

The guest's credential-release unit requires both /dev/disk/by-label/opkeydata and the server role. An unmet condition makes systemd skip the unit rather than fail it, so a node without the disk boots and attests while nothing listens on 8443.

The joindata disk

An ISO9660 filesystem whose volume label is joindata, mounted read-only. Each field is one file holding a single ASCII line of at most 256 bytes. Leading and trailing whitespace is trimmed; interior whitespace, NUL bytes, extra lines, symlinks, and files outside the role's list are rejected, and a rejected disk fails the role unit and every unit gated on it.

FileValueServerAgent
roleserver or agentrequiredrequired
node-nameRFC1123 labeloptionaloptional
node-ipIPv4optionaloptional
node-external-iproutable IPv4optionaloptional
serverserver's IPv4, no scheme or portrejectedrequired
server-token64 lowercase hex charactersrequiredrejected
agent-token64 lowercase hex charactersrequiredrequired
  • server-token and agent-token must differ.
  • An absent node-name keeps the image's baked hostname, c8s-node. Every node past the first needs its own.
  • An absent node-ip, or the value 0.0.0.0, leaves address selection to RKE2. 0.0.0.0 in node-external-ip is an error.
  • An agent dials https://<server>:9345.
  • With no disk the node is a single-node server with a boot-local agent token.

Network ports

PortDirectionService
6443inboundkube-apiserver
8400inboundattestation-api: /health, /attest
8443inboundcredential release, server role only
9345inboundRKE2 supervisor, dialed by agents
443inboundtls-lb front door, after c8s install
1019inboundadmission-inventory digests endpoint, dialed by the CDS
outboundNTP for systemd-timesyncd, and the default route the inventory advertises

The guest needs a default route: the admission inventory takes the address it advertises to the CDS from it, and refuses to start with a loopback address. Clock drift trips the token check on c8s get-kubeconfig, which is why outbound NTP matters.

Baked cluster settings

Set in the image and covered by its launch measurement. Runtime overrides go in /etc/rancher/rke2/config.yaml.d/, and RKE2 fixes cluster and service CIDRs at first initialization.

SettingValue
hostnamec8s-node
CNICilium, with disable-kube-proxy: true
cluster CIDR10.52.0.0/16
service CIDR10.53.0.0/16
cluster DNS10.53.0.10
apiserver SANc8s-cvm — the default for c8s get-kubeconfig --tls-server-name
disabled componentsrke2-ingress-nginx, which leaves host port 443 to tls-lb
hardening profilecis, with pod security admission and apiserver audit logging
kubeletmax-pods=200
node labelconfidential.ai/sev-snp=true or confidential.ai/tdx=true, from the build platform

The platform label is what c8s install --hardware-platform selects on, and the node applies it itself at boot.

Manifest fields c8s pins

manifest.json ships in the artifact and is the input to c8s get-kubeconfig --image-manifest and c8s measurements derive. Its shape selects the platform.

FieldPlatformMeaning
versionbothmanifest schema version; c8s reads version 3
build.platformbothsnp or tdx
build.memorybothguest RAM the build was validated at
snp_variants[].smpSEV-SNPvCPU count of one measured variant
snp_variants[].igvm.pathSEV-SNPthe IGVM file for that count
snp_variants[].measurement.snp_launch_digestSEV-SNPSHA-384 launch digest for that count
snp_variants[].measurement.algorithmSEV-SNPmust be sha384
tdx.mrtdIntel TDXlaunch measurement of the TDVF firmware
tdx.rtmr1, tdx.rtmr2Intel TDXguest kernel and root filesystem
tdx.firmware.pathIntel TDXthe TDVF file the tuple describes

An SEV-SNP image pin is the snp_variants set of one build manifest, and a file without that field is rejected. Duplicate smp entries fail the whole load, so a policy pins an image whole. RTMR[3] is a deployment property — the operator key, extended by any measured workload images — so it comes from c8s get-kubeconfig flags rather than from the manifest.

See also