Start policy enforcement

Start policy enforcement before launching workers. The Certificate Distribution Service (CDS) is the policy engine: it holds operator-approved policy that measured workers retrieve and enforce locally.

Its three confidential VMs share a mesh CA, allowlist, and secret store through a replicated log. CDS runs independently of Kubernetes and takes no Kubernetes credentials. You can host it alongside workers or elsewhere with compatible confidential hardware and network reachability.

Before you start

Complete Prepare launch inputs. Keep its working directory and use the same shell variables and approved artifacts.

Have three VM slots ready with the selected TEE and a primary IPv4 address per VM. Prepare DNS and firewall rules:

• cds-1.example.com, cds-2.example.com, and cds-3.example.com resolve directly to their replicas. Discovery names must reach the replicas themselves.
• Every replica reaches the other replicas on TCP 8444, used for attested peer traffic and HA administration.
• Workers and operators reach the CDS client endpoint on TCP 8443. The operator machine also reaches each replica directly on TCP 8444.
• Each replica can resolve DNS and reach the CPU vendor's endorsement-certificate/collateral services needed for attestation.
• cds.example.com:8443 reaches ready replicas through DNS or a TLS-passthrough load balancer. Use HTTPS GET /readyz for balancer membership and preserve the attested TLS connection to CDS.

A cloud or bare-metal launcher must satisfy the measured launch contract below. The c8s commands configure and initialize CDS; your VM substrate boots it.

Build the read-only launch disk

Every replica boots the same exact cds/cds-config.yaml bytes. Put them in a root file named pubkey on an ISO9660 disk labelled opkeydata:

mkdir cds-opkeydata
cp cds/cds-config.yaml cds-opkeydata/pubkey
cmp cds/cds-config.yaml cds-opkeydata/pubkey
sha256sum cds-opkeydata/pubkey
genisoimage -quiet -rock -joliet -volid opkeydata \
  -output cds-opkeydata.iso cds-opkeydata/

The file name pubkey is the launch-disk interface: its contents are the complete CDS configuration. Keep operator.key off this disk.

For TDX, boot the approved artifacts/cds/disk.raw with the matching TDX firmware and boot artifacts from that build. The measured initrd extends the configuration's SHA-384 into RTMR[3], a hardware runtime measurement register.

For SEV-SNP, boot the manifest's approved guest-smp<N>.igvm variant with the matching vCPU count and disk. Set the launcher's HOST_DATA to the SHA-256 of cds-opkeydata/pubkey printed above. This hashes the file's bytes, not the ISO or filesystem. The initrd checks that binding before starting CDS.

Attach cds-opkeydata.iso read-only to all three VMs. CDS needs no scratch disk: its state is held in guest memory. Assign each VM its planned IPv4 address, connect its network, and boot it using your substrate's TEE launcher. Retain the VM launch records and artifact digests.

The measured guest stages pubkey at /run/confai/operator-pubkey. Its systemd service starts CDS with that configuration and verifies that a fresh hardware report binds the exact parsed bytes. CDS listens on the guest's primary IPv4 address.

Check each replica before Genesis

Run from your trusted operator machine:

if ! c8s cds ha status \
  --config cds/cds-config.yaml --cds-pin cds/cds-pin.json; then
  printf '%s\n' 'Inspect all three authenticated replica responses before continuing.'
fi

The command authenticates each replica against the locally approved CDS image/config pin. Before Genesis it exits nonzero because the replicas are not serving a shared cluster. Inspect all three authenticated responses and continue only when every replica reports joined=false serving=false.

If a replica is unreachable, inspect the substrate's VM console and network:

• Check the launch disk label and file, and the exact configuration binding.
• Check the primary IPv4 address, peer port 8444, DNS, and endorsement-certificate access.
• For TDX, check host quote generation and platform registration.

Resolve these failures before signing Genesis.

Initialize the shared cluster

Genesis is the signed instruction that creates this cluster. It admits the first worker image together with its exact launch document. Use the files prepared before booting CDS:

c8s cds ha decree genesis \
  --config cds/cds-config.yaml --operator-key operator.key \
  --target cds-1.example.com --cds-pin cds/cds-pin.json \
  --platform tdx --image-manifest cds-manifest.json \
  --resume-version 0 \
  --worker-document worker.yaml --worker-image worker-image.json \
  --out genesis.json

For SEV-SNP use --platform sev-snp --vcpus 4 with the approved four-vCPU CDS image variant, matching the pin generated earlier.

The CLI prints the worker's API endpoint, CA fingerprint, cluster ranges, CDS endpoint, router settings, and launch digest. Compare those facts with your control-plane records and approved documents. Confirm interactively only when they match. The command signs with operator.key and posts the decree to the named replica over the pinned attested connection.

--resume-version 0 is for the first cluster under this cluster ID. Preserve the highest version reached in your deployment record for any future recovery. The three-replica setup uses peer discovery during Genesis.

Verify the quorum and served policy

Wait for the other replicas to join, then run:

c8s cds ha status \
  --config cds/cds-config.yaml --cds-pin cds/cds-pin.json
c8s cds ha decrees \
  --config cds/cds-config.yaml --cds-pin cds/cds-pin.json
c8s cds ha pin-policy \
  --config cds/cds-config.yaml --cds-pin cds/cds-pin.json \
  --out cds/served-pin.json --decree-out cds/served-pin-decree.json
c8s measurements lint cds/served-pin.json

Require successful commands and three status lines with:

• joined=true serving=true on every replica.
• One state="Leader", with voters=3 and roll_safe=true on its line.
• The same cluster ID, version=1, and ca= fingerprint on all three.
• No ALARM lines once the replicas have caught up.

The decree record must contain the signed Genesis. pin-policy verifies its operator signature, cluster ID, and launch binding before writing the served CDS image policy. At this initial stage, compare its measurement entries with cds/cds-pin.json; both must admit the same image and configuration. Retain the original pin as your locally approved input.

Read the shared CA through the verified client endpoint and compare the live allowlist with the seed:

c8s cds ca --url https://cds.example.com:8443 \
  --image-policy-file cds/cds-pin.json --out mesh-ca.pem
c8s allowlist diff render/allowlist-seed.json --exit-code \
  --url https://cds.example.com:8443 \
  --image-policy-file cds/cds-pin.json

Check both results:

• The CA fingerprint printed by cds ca must match the status fingerprint. Record it.
• The allowlist diff must print no changes.

Genesis seeds the platform entries and certifies the worker image bound to worker.yaml. For a machine-readable record of that admission, save the verified decrees:

c8s cds ha decrees --json \
  --config cds/cds-config.yaml --cds-pin cds/cds-pin.json \
  > cds/applied-decrees.json

Keep the Genesis worker image set and launch digest with your approval record. Wait 30 seconds after Genesis before the first operator policy write, matching the default token clock window.

Keep the trust root available

Three voting replicas tolerate one lost replica while a majority remains connected and holds its serving lease. Before a planned replacement, require roll_safe=true from the leader. Replace one VM at a time and wait for it to rejoin and for that gate to return before replacing another.

CDS replication preserves availability, but its CA, secrets, and replicated state are volatile. Terminating every replica loses that state. A customer-key encrypted CDS backup/export is not available in this source revision. Keep a running quorum throughout routine maintenance.

Before continuing, confirm you have:

• A reachable CDS endpoint at cds.example.com:8443 and a healthy replicated trust root.
• The approved cds/cds-pin.json and mesh-ca.pem.
• worker.yaml, worker-image.json, and artifacts/worker/ from preparation.
• The component digests and external control-plane credentials from preparation.

Keep all launch and approval records together.