Pin multiple measurements with a measurements config

Write one JSON file listing every VM image a cluster accepts, derive it from the image build manifests, install with it, and prove the cluster enforces exactly that set.

A measurements config is one JSON file listing the VM images a cluster accepts. c8s matches an entry as a whole — the launch digest together with the runtime registers measured from the same build — so a digest from one build can never be paired with another build's registers. This guide derives that file, installs with it, and checks that the cluster is enforcing exactly what you pinned. For the trust argument behind whole-image matching, see the verification model.

When you need this

  • One AMD SEV-SNP build already has several launch digests. An SEV-SNP launch digest covers the initial vCPU state, so a build ships one measurement per vCPU count and c8s measurements derive emits one entry per variant. A single-image SNP cluster is already a multi-entry pin.
  • A rolling image upgrade. List the outgoing image and the incoming one in the same file, roll the fleet, then re-derive from the new image alone. The cluster stays pinned the whole way through.

Before you start

  • The build output directory of each VM image the cluster runs — the directory holding that image's manifest.json. Fetch a manifest over the same channel you trust for the image itself: derive reads the manifest, it does not re-measure the image bytes.
  • Every image built for the same platform. One file describes one platform, and a cluster mixing SEV-SNP and Intel TDX images is not supported.
  • The c8s CLI on PATH, kubectl configured for the cluster, and network reach to the CDS endpoint for the final check.

What the file looks like

Three top-level fields: schema_version, always "1"; tee, either "sev-snp" or "tdx"; and measurements, an array of at least one entry. Every register is 96 lowercase hex characters — a SHA-384 value.

An SEV-SNP file, one image built for four vCPU counts:

{
  "schema_version": "1",
  "tee": "sev-snp",
  "measurements": [
    {
      "name": "c8s-worker-snp-smp2",
      "measurement": "e7df3a8f1dbe619607154ce994c1f4d7299c539b120b5560e137f7787e4ece304f270c1444b47c863fde54bc863291d7"
    },
    {
      "name": "c8s-worker-snp-smp4",
      "measurement": "e0615da999afd20375d64ec9aaf0c480d6854198c01ed3e4157f881a96f635e532f7ad528ef69b408ef10319be5db106"
    },
    {
      "name": "c8s-worker-snp-smp8",
      "measurement": "7e32c5c22d2eee8044d0206ec183783fb1c442a4089cf41491a2c3e1805ec99a7e571eca98eddd80d521e69716033157"
    },
    {
      "name": "c8s-worker-snp-smp16",
      "measurement": "3d77864bca73abbe93a00192e39daf23abe1d8a95b059e12d0189f8c17c6af3144be01713c6d6def0eb858361701ea5b"
    }
  ]
}

measurement is the SEV-SNP launch digest, which already folds firmware, kernel, initrd, and the initial vCPU state into one value. name identifies the entry in diagnostics; it is unique per file and carries no matching semantics.

A TDX file, one image pinned as its full tuple:

{
  "schema_version": "1",
  "tee": "tdx",
  "measurements": [
    {
      "name": "c8s-worker-tdx",
      "mrtd": "9309eaae9c151e766de0f97b1d1aaeb76b8c8c366080803943fb566521c8f0cf00a142d8b7b0683ed1d42c5a27198ba1",
      "rtmr": [
        null,
        "fc0e743e04e2d7531c8b0e4168f5814ac769905bedd75333b64984a84f33082be89125a523df2febe9ec85ed912719a9",
        "deda5c913487bf3cde10497c260ce2333267fa107c98740ef7849a74034911a29e9d1327b1e8a16b44664079ec62a8d6",
        null
      ]
    }
  ]
}

On TDX the launch digest is mrtd, and rtmr is a positional array of four:

  • rtmr[0] must be null. It carries the TD HOB, so it varies with the vCPU and memory shape of the VM. A file that pins it is rejected outright.
  • rtmr[1] and rtmr[2] are the guest image: the guest kernel, and the kernel command line carrying the dm-verity root hash. MRTD covers the TDVF firmware, so these two are what make the guest's own bytes attested.
  • rtmr[3] is the runtime chain — the operator-key seed extended at launch, then a workload image digest per deployed workload. An image build does not pin it.

Init-data is a different binding and does not appear in this file: the Kata shim commits it to SEV-SNP HOST_DATA and TDX MRCONFIGID, and c8s verify --init-data pins it there.

The parser is strict — lowercase hex only, no unknown fields, no duplicate JSON keys, no duplicate names, and no two entries pinning the same tuple. Two entries may share an MRTD and differ in their registers; that is exactly the case the tuple exists for.

Pin the images

Derive the file from the image manifests

c8s measurements derive ./out/c8s-worker-snp --out measurements.json

The document goes to --out (or to stdout when it is omitted); progress goes to stderr:

c8s-worker-snp: 4 sev-snp entries from out/c8s-worker-snp/manifest.json

Entry names come from the directory the image was built into — this build produced c8s-worker-snp-smp2, -smp4, -smp8 and -smp16. A TDX image contributes one entry carrying its MRTD with RTMR[1] and RTMR[2]:

c8s measurements derive ./out/c8s-worker-tdx --out measurements.json
c8s-worker-tdx: 1 tdx entry from out/c8s-worker-tdx/manifest.json

Pass several images in one invocation to pin them all in one file. They must share a platform, and an image built for both requires --tee sev-snp or --tee tdx:

c8s measurements derive ./out/c8s-worker-tdx-v1 ./out/c8s-worker-tdx-v2 --out measurements.json

Check the file

c8s measurements lint measurements.json
ok: sev-snp, 4 image(s)
  c8s-worker-snp-smp2: 0 register pin(s)
  c8s-worker-snp-smp4: 0 register pin(s)
  c8s-worker-snp-smp8: 0 register pin(s)
  c8s-worker-snp-smp16: 0 register pin(s)

lint loads the file exactly as every component loads it, so a file that lints clean is the file the cluster will run on. SNP entries report 0 register pin(s): the launch digest is the whole pin there. A problem exits non-zero and names the entry and field:

measurements config measurements.json: measurements[0].rtmr[0]: must be null — RTMR[0] varies with vCPU and memory shape

Install with it

c8s install --cvm-mode=node --hardware-platform=sev-snp \
  --operator-keys operator.pub \
  --measurements-config measurements.json

--cvm-mode and --hardware-platform are both required; the platform here is the one the file's tee field names.

The installer validates the file, sends its content to the chart as cds.measurementsConfig and ratlsMesh.measurementsConfig, and fans the same pins out flat as cds.measurements and cds.rtmrs (plus the ratlsMesh pair) for the components that read a digest list. --measurements-config cannot be combined with --measurements or --rtmrs — that pair is a usage error, not a merge. Every flag is in the CLI reference.

Two install messages read the flat flags only.

A config-mode install still prints TDX with no --rtmrs on TDX and the mesh is UNPINNED hint at the end, because both texts branch on --measurements and --rtmrs. Confirm the pin from the rendered arguments in the next step rather than from those lines.

Read the pin back off the cluster

The chart renders the document into the c8s-measurements ConfigMap and mounts it read-only at /etc/c8s-measurements in cds, ratls-mesh, operator, and tls-lb:

kubectl get configmap c8s-measurements -n c8s-system -o jsonpath='{.data.cds\.json}'

Confirm CDS is running on it:

kubectl get deploy c8s-cds -n c8s-system -o yaml | grep -- --measurements
            - --measurements-config=/etc/c8s-measurements/cds.json

Each of those four pods carries a checksum/measurements-config annotation over the same content, so a changed pin rolls them. The components hold the file they loaded at startup — change the pin by re-running c8s install, and let the annotation roll the pods.

Prove the cluster enforces exactly this file

The CDS serves the reference values it is enforcing at GET /measurements, built from the values it resolved at startup. c8s verify --measurements-config fetches that document for a cds target and requires an exact match in both directions:

CDS_IP=$(kubectl get endpoints c8s-cds -n c8s-system -o jsonpath='{.subsets[0].addresses[0].ip}')

c8s cds verify "https://$CDS_IP:8443" --measurements-config measurements.json

The fetch runs on a connection pinned to the serving certificate the attestation step already verified, so a substituted endpoint cannot answer for the CDS on the second connection. Any difference fails the run with exit code 2 and names the image:

  reason:       the target admits an image --measurements-config does not pin: c8s-worker-snp-smp8 (7e32c5c22d2eee8044d0206ec183783fb1c442a4089cf41491a2c3e1805ec99a7e571eca98eddd80d521e69716033157)

The mirror-image failure — --measurements-config pins an image the target does not admit — means the cluster is enforcing less than the file says.

A target that pins nothing fails too — it serves an empty set, which reads as admits any TEE attestation. So does a target that predates the endpoint, and so does a fetch that errors: the check never passes unchecked. A ✓ VERIFIED verdict means the served set is exactly your file. The verdict's measurement_pinned field and its WARNING: no --measurements pinned line track --measurements, so read the exit code and the reason: line for this check.

Roll a new image in

Derive both images into one file, lint it, and re-run the install with it. Both images attest for as long as both entries are listed, so nodes move one at a time with no pinning gap:

c8s measurements derive ./out/c8s-worker-tdx-v1 ./out/c8s-worker-tdx-v2 --out measurements.json
c8s measurements lint measurements.json
c8s install --cvm-mode=node --hardware-platform=tdx \
  --operator-keys operator.pub --measurements-config measurements.json

Once every node runs the new image, derive again from that image alone, re-install, and re-run the c8s cds verify --measurements-config check. The cluster then admits one image again.

What one file covers

  • One platform. tee is sev-snp or tdx, and the CDS refuses to start when the file's tee disagrees with --ratls-platform; ratls-mesh refuses when it disagrees with the platform the node attests as. Mixed SNP and TDX fleets are not supported.
  • Every role. The same file names the images that may call /attest and the CDS identity the other components dial, so any image listed in it may serve as the CDS.
  • Whole images, where a gate can express one. The CDS /attest path, ratls-mesh (peers and its CDS dial), and the allowlist-proxy match entries whole. The components that carry a plain digest list — nri-image-policy, injected sidecars, CA handoff, /attest-key — take the digests, plus the register pins when every entry pins the same registers. When TDX entries pin different registers, those gates pin digests only, and each logs a warning at startup naming that.

See also