Your first confidential cluster

Prepare a measured external CDS, install c8s on an existing confidential AKS cluster, and verify the router against approved image and deployment pins.

This tutorial takes an existing confidential AKS cluster through its first c8s installation. You'll prepare the external Certificate Distribution Service (CDS), install the cluster components, and check the router from your workstation using independently approved pins.

The nodes use AMD SEV-SNP, a Trusted Execution Environment (TEE) that protects guest memory from the host. Each node is one confidential VM (CVM), with its pods sharing that trust domain. The CDS runs in a separate CVM. Kubernetes readiness and a valid hardware report alone do not establish that you're running the code you intended.

Prerequisites

Prepare these inputs before starting:

  • An AKS cluster running Kubernetes 1.30 or newer on SEV-SNP confidential nodes, with kubectl already configured. For cloud provisioning, use your validated Azure procedure and the AKS provisioning guide; follow this tutorial for the c8s bootstrap. Confirm SKU availability, pool support, and node-image compatibility for your subscription.
  • A Kubernetes identity authorized to create and update c8s cluster-scoped resources, including CRDs, admission webhooks, ClusterRoles, and ClusterRoleBindings, and to create and update component namespaces and workloads. This includes setting privileged Pod Security labels on c8s-system.
  • Every Node must publish at least one pod range in spec.podCIDR or spec.podCIDRs.
  • The c8s CLI, helm, kubectl, crane, openssl, sha256sum, and genisoimage on a Linux workstation. This tutorial documents c8s revision 0345c5140cc7e1e5f7a77edb096f59f532873b2b; building it requires Go 1.27.3 or a compatible newer toolchain. See Install c8s for the build command.
  • A component image tag compatible with that CLI and chart, accessible from your workstation and nodes. Substitute it for <IMAGE_TAG> in both seed and install commands. Confirm the artifacts exist before starting; a source revision does not establish registry availability.
  • Approved node launch identities saved as nodes-policy.json, and an approved policy for the router's nodes saved as router-pin.json. These are complete JSON image policies from your trusted image and deployment records. Reuse the same policy only if its entire node set is acceptable for the router. See Obtain launch measurements.
  • The cluster's node, pod, and service CIDRs. Node inventory subnets must exclude pod and service ranges. Substitute these for <NODE_CIDR>, <POD_CIDR>, and <SERVICE_CIDR> below.
  • A complete SEV-SNP CDS build bundle from your image producer: disk.raw, manifest.json, and the matching guest-smp<N>.igvm. <CDS_MANIFEST_JSON> names that manifest and <CDS_VCPU_COUNT> selects its supported vCPU count. Have a VM provisioning mechanism ready that boots this bundle, attaches an opkeydata ISO, and sets SNP HOST_DATA. Obtain its launch instructions, including the input encoding for the 32-byte HOST_DATA field.
  • A network plan allowing the workstation and cluster to reach CDS on TCP 8443, and CDS to reach admitted nodes' inventory endpoints on TCP 1019. <CDS_HOST> is the CDS VM's reachable DNS name or IPv4 address. Provide vendor verification collateral through evidence, a local cache, or network retrieval when required.
  • A network path from your workstation to the node hosting the router on TCP 443. The default router exposes this host port; step 6 identifies the hosting node and its address. Provide private-network access if that node has only an internal address. This tutorial keeps the default certificate name c8s-router.c8s-router.svc and TLS mode; any intervening load balancer must pass TLS through to the router.

Use a fresh local directory for the two policy files and the generated files below. The commands keep the default release c8s and namespace c8s-system. This path creates a singleton CDS; the VM launch mechanism is a required external preparation, not a Kubernetes resource created by installation.

Step by step

1. Check the target cluster

kubectl config current-context
kubectl version
kubectl get nodes -o wide
kubectl get nodes \
  -o custom-columns='NAME:.metadata.name,PODCIDR:.spec.podCIDR,PODCIDRS:.spec.podCIDRs'

Each row must have a pod CIDR in at least one of the two pod-range columns. Stop if any Node has neither and resolve its pod-range publication before generating the seed or preparing CDS. Supplying <POD_CIDR> in the CDS configuration does not satisfy this Node-field check.

Confirm the context names your prepared cluster, the server meets the version floor, and its confidential nodes are ready. --cvm-mode aks selects the Azure vTPM evidence path; --hardware-platform sev-snp selects its CPU TEE. The node list is a health check, not attestation of those nodes.

2. Create the operator credential and component seed

umask 077
openssl ecparam -name prime256v1 -genkey -noout -out operator.key
openssl ec -in operator.key -pubout -out operator.pub

c8s install --cvm-mode aks --hardware-platform sev-snp \
  --image-tag '<IMAGE_TAG>' --operator-keys operator.pub \
  --allowlist-seed-out component-seed.json

Keep operator.key private. Its public half will authorize policy writes through the measured CDS configuration. The installer's --operator-keys also supplies router verification keys; it does not configure the authority of an already running CDS.

The seed command reads your cluster and component registries, writes component-seed.json, and stops before installation. This file contains the component entries that the image allowlist must admit. Keep the image tag and install options unchanged through the final install.

3. Prepare and launch the external CDS

c8s cds config new --out cds-bootstrap --cluster-id first-cluster \
  --operator-keys operator.pub --allowlist-seed component-seed.json \
  --node-pins nodes-policy.json \
  --node-cidr '<NODE_CIDR>' --pod-cidr '<POD_CIDR>' --service-cidr '<SERVICE_CIDR>' \
  --image-manifest '<CDS_MANIFEST_JSON>' --platform sev-snp \
  --vcpus '<CDS_VCPU_COUNT>'

Use the prepared ranges, manifest, and vCPU count. Repeat a CIDR flag if your cluster has multiple ranges. first-cluster is the cluster identity you assign this configuration. The new directory contains cds-config.yaml and cds-pin.json; the pin binds the CDS image and the exact configuration bytes. Keep private keys and secrets out of this host-readable configuration.

Package the configuration as the VM's input disk:

mkdir cds-opkeydata
cp cds-bootstrap/cds-config.yaml cds-opkeydata/pubkey
sha256sum cds-opkeydata/pubkey
genisoimage -quiet -rock -joliet -volid opkeydata \
  -output cds-opkeydata.iso cds-opkeydata/

Launch the CDS with your prepared VM provisioning mechanism. Attach cds-opkeydata.iso, use the matching disk and IGVM from the approved build, set the selected vCPU count, and set HOST_DATA to the 32 raw bytes of the SHA-256 digest. sha256sum prints those bytes as the first 64 hexadecimal characters, followed by the filename. Follow your provisioner's launch instructions to encode those digest bytes in its input format; the filename is not part of HOST_DATA. The guest stages the configuration at /run/confai/operator-pubkey; CDS checks it against its own launch evidence before serving. Wait for the VM's network endpoint to become reachable before proceeding.

4. Verify CDS and retain the mesh CA

c8s cds verify 'https://<CDS_HOST>:8443' \
  --image-policy-file cds-bootstrap/cds-pin.json \
  --served-policy-file nodes-policy.json --operator-keys operator.pub && \
c8s cds ca --url 'https://<CDS_HOST>:8443' \
  --image-policy-file cds-bootstrap/cds-pin.json --out mesh-ca.pem

Replace <CDS_HOST> with the external VM endpoint. Require verification exit code 0; && prevents the CA read after any other verdict. Continue only after the CA command also succeeds. If verification cannot retrieve required collateral, resolve that failure first.

This checks three separate inputs: the CDS image and configuration, the intended node set it reports admitting, and the expected operator keys. The authenticated CA read saves mesh-ca.pem, the deployment anchor for certificates CDS issues. Keep it with your approved pins and compare later CA reads before replacing it.

Keep the singleton running.

Its CA and secrets live in memory. Restarting it replaces that volatile state. Keep secret escrow and follow CDS lifecycle guidance before restarting.

5. Install and check the cluster components

c8s install --cvm-mode aks --hardware-platform sev-snp \
  --image-tag '<IMAGE_TAG>' --operator-keys operator.pub \
  --cds-address '<CDS_HOST>:8443' --cds-pin cds-bootstrap/cds-pin.json

Use the same image tag as the seed command. The installer checks the configured component seed and node subnets, then runs Helm with readiness waiting enabled. Require successful completion before checking the resulting objects:

kubectl get pods -n c8s-system
kubectl get pods -n c8s-router
kubectl get mutatingwebhookconfiguration c8s-pod-injector \
  -o jsonpath='{.webhooks[*].clientConfig.caBundle}'

Expect component pods to be ready and the webhook CA bundle to be nonempty. The operator, attestation API, and Node Resource Interface (NRI) image-policy installer/worker serve the cluster; the router runs in c8s-router. CDS remains in its external VM. Every newly admitted pod in a namespace matched by the webhook's namespaceSelector receives its own c8s-mesh endpoint, with NRI enforcing nftables packet rules. See the namespace coverage check before selecting an application namespace.

On managed nodes, validate the containerd changes and privileged node components through node-image upgrades. If installation fails, inspect the pods and events, correct the cause, and rerun the same pinned install command. Readiness establishes health; the next check establishes the router's attested identity.

6. Verify the router from your workstation

Identify the node hosting a ready router pod:

kubectl get pods -n c8s-router -o wide
kubectl get nodes -o wide

Match the router pod's NODE column to the node list. Set <ROUTER_ADDRESS> in the next command to that node's IP address reachable from your workstation. The default host port serves the router on TCP 443. If only an internal address is listed, use the private-network path prepared above. Stop and establish that path if the address is unreachable. Pod port-forwarding is not a substitute: packet rules reject root-owned sockets entering a pod's network namespace.

c8s verify 'https://<ROUTER_ADDRESS>:443' --kind lb \
  --server-name c8s-router.c8s-router.svc \
  --image-policy-file router-pin.json --mesh-ca mesh-ca.pem

Replace <ROUTER_ADDRESS> with the reachable router address. --server-name supplies the default certificate's DNS name even when connecting to an IP address. router-pin.json checks the router node's approved identity; mesh-ca.pem checks its deployment's issuing CA. The CDS image pin belongs to a different VM and cannot substitute for the router policy.

Require exit code 0 and inspect the verdict's warnings. Discovery binds the certificate to issuance-time evidence, so expect fresh: false: it does not contain a nonce from this verification attempt. The check also compares the live TLS serving certificate with the attested certificate. Its result covers this connection to one router replica at one instant. It does not verify an application's later connection or backend workload.

Exit 1 means invalid inputs, 2 means verification or policy failure, 3 means required evidence or network material was unavailable, and 4 means partial verification. None is a successful completion of this tutorial. See Verify a deployment for verdict fields and fresh route checks.

What you've built

You have a c8s installation whose components trust an external, launch-pinned CDS, a retained mesh CA, and a router checked against its own approved node policy. You have also checked Kubernetes readiness separately from attested identity.

Where to go next

Run a confidential workload adds an application, admits its complete container policy, publishes a signed route, and sends a request through a verified encrypted session. The application serves ordinary HTTP inside its pod; injected platform containers hold the private mesh credentials.

For the trust assumptions behind these steps, read trust boundaries and limitations.