Your first confidential cluster
Prepare a measured external CDS, install c8s on an existing confidential AKS cluster, and verify the router against approved image and deployment pins.
This tutorial takes an existing confidential AKS cluster through its first c8s installation. You'll prepare the external Certificate Distribution Service (CDS), install the cluster components, and check the router from your workstation using independently approved pins.
The nodes use AMD SEV-SNP, a Trusted Execution Environment (TEE) that protects guest memory from the host. Each node is one confidential VM (CVM), with its pods sharing that trust domain. The CDS runs in a separate CVM. Kubernetes readiness and a valid hardware report alone do not establish that you're running the code you intended.
Prerequisites
Prepare these inputs before starting:
- An AKS cluster running Kubernetes 1.30 or newer on SEV-SNP confidential nodes, with
kubectlalready configured. For cloud provisioning, use your validated Azure procedure and the AKS provisioning guide; follow this tutorial for the c8s bootstrap. Confirm SKU availability, pool support, and node-image compatibility for your subscription. - A Kubernetes identity authorized to create and update c8s cluster-scoped resources,
including CRDs, admission webhooks, ClusterRoles, and ClusterRoleBindings, and to create
and update component namespaces and workloads. This includes setting privileged Pod Security
labels on
c8s-system. - Every Node must publish at least one pod range in
spec.podCIDRorspec.podCIDRs. - The
c8sCLI,helm,kubectl,crane,openssl,sha256sum, andgenisoimageon a Linux workstation. This tutorial documents c8s revision0345c5140cc7e1e5f7a77edb096f59f532873b2b; building it requires Go 1.27.3 or a compatible newer toolchain. See Install c8s for the build command. - A component image tag compatible with that CLI and chart, accessible from your workstation
and nodes. Substitute it for
<IMAGE_TAG>in both seed and install commands. Confirm the artifacts exist before starting; a source revision does not establish registry availability. - Approved node launch identities saved as
nodes-policy.json, and an approved policy for the router's nodes saved asrouter-pin.json. These are complete JSON image policies from your trusted image and deployment records. Reuse the same policy only if its entire node set is acceptable for the router. See Obtain launch measurements. - The cluster's node, pod, and service CIDRs. Node inventory subnets must exclude pod and
service ranges. Substitute these for
<NODE_CIDR>,<POD_CIDR>, and<SERVICE_CIDR>below. - A complete SEV-SNP CDS build bundle from your image producer:
disk.raw,manifest.json, and the matchingguest-smp<N>.igvm.<CDS_MANIFEST_JSON>names that manifest and<CDS_VCPU_COUNT>selects its supported vCPU count. Have a VM provisioning mechanism ready that boots this bundle, attaches anopkeydataISO, and sets SNPHOST_DATA. Obtain its launch instructions, including the input encoding for the 32-byteHOST_DATAfield. - A network plan allowing the workstation and cluster to reach CDS on TCP
8443, and CDS to reach admitted nodes' inventory endpoints on TCP1019.<CDS_HOST>is the CDS VM's reachable DNS name or IPv4 address. Provide vendor verification collateral through evidence, a local cache, or network retrieval when required. - A network path from your workstation to the node hosting the router on TCP
443. The default router exposes this host port; step 6 identifies the hosting node and its address. Provide private-network access if that node has only an internal address. This tutorial keeps the default certificate namec8s-router.c8s-router.svcand TLS mode; any intervening load balancer must pass TLS through to the router.
Use a fresh local directory for the two policy files and the generated files below. The
commands keep the default release c8s and namespace c8s-system. This path creates a
singleton CDS; the VM launch mechanism is a required external preparation, not a Kubernetes
resource created by installation.
Step by step
1. Check the target cluster
kubectl config current-context
kubectl version
kubectl get nodes -o wide
kubectl get nodes \
-o custom-columns='NAME:.metadata.name,PODCIDR:.spec.podCIDR,PODCIDRS:.spec.podCIDRs'Each row must have a pod CIDR in at least one of the two pod-range columns. Stop if any Node
has neither and resolve its pod-range publication before generating the seed or preparing
CDS. Supplying <POD_CIDR> in the CDS configuration does not satisfy this Node-field check.
Confirm the context names your prepared cluster, the server meets the version floor, and
its confidential nodes are ready. --cvm-mode aks selects the Azure vTPM evidence path;
--hardware-platform sev-snp selects its CPU TEE. The node list is a health check, not
attestation of those nodes.
2. Create the operator credential and component seed
umask 077
openssl ecparam -name prime256v1 -genkey -noout -out operator.key
openssl ec -in operator.key -pubout -out operator.pub
c8s install --cvm-mode aks --hardware-platform sev-snp \
--image-tag '<IMAGE_TAG>' --operator-keys operator.pub \
--allowlist-seed-out component-seed.jsonKeep operator.key private. Its public half will authorize policy writes through the measured
CDS configuration. The installer's --operator-keys also supplies router verification keys;
it does not configure the authority of an already running CDS.
The seed command reads your cluster and component registries, writes component-seed.json,
and stops before installation. This file contains the component entries that the
image allowlist must admit. Keep the image tag and install
options unchanged through the final install.
3. Prepare and launch the external CDS
c8s cds config new --out cds-bootstrap --cluster-id first-cluster \
--operator-keys operator.pub --allowlist-seed component-seed.json \
--node-pins nodes-policy.json \
--node-cidr '<NODE_CIDR>' --pod-cidr '<POD_CIDR>' --service-cidr '<SERVICE_CIDR>' \
--image-manifest '<CDS_MANIFEST_JSON>' --platform sev-snp \
--vcpus '<CDS_VCPU_COUNT>'Use the prepared ranges, manifest, and vCPU count. Repeat a CIDR flag if your cluster has
multiple ranges. first-cluster is the cluster identity you assign this configuration.
The new directory contains cds-config.yaml and cds-pin.json; the pin binds the CDS image
and the exact configuration bytes. Keep private keys and secrets out of this host-readable
configuration.
Package the configuration as the VM's input disk:
mkdir cds-opkeydata
cp cds-bootstrap/cds-config.yaml cds-opkeydata/pubkey
sha256sum cds-opkeydata/pubkey
genisoimage -quiet -rock -joliet -volid opkeydata \
-output cds-opkeydata.iso cds-opkeydata/Launch the CDS with your prepared VM provisioning mechanism. Attach cds-opkeydata.iso, use
the matching disk and IGVM from the approved build, set the selected vCPU count, and set
HOST_DATA to the 32 raw bytes of the SHA-256 digest. sha256sum prints those bytes as the
first 64 hexadecimal characters, followed by the filename. Follow your provisioner's launch
instructions to encode those digest bytes in its input format; the filename is not part of
HOST_DATA. The guest stages the configuration at /run/confai/operator-pubkey; CDS checks
it against its own launch evidence before serving.
Wait for the VM's network endpoint to become reachable before proceeding.
4. Verify CDS and retain the mesh CA
c8s cds verify 'https://<CDS_HOST>:8443' \
--image-policy-file cds-bootstrap/cds-pin.json \
--served-policy-file nodes-policy.json --operator-keys operator.pub && \
c8s cds ca --url 'https://<CDS_HOST>:8443' \
--image-policy-file cds-bootstrap/cds-pin.json --out mesh-ca.pemReplace <CDS_HOST> with the external VM endpoint. Require verification exit code 0;
&& prevents the CA read after any other verdict. Continue only after the CA command also
succeeds. If verification cannot retrieve required collateral, resolve that failure first.
This checks three separate inputs: the CDS image and configuration, the intended node set
it reports admitting, and the expected operator keys. The authenticated CA read saves
mesh-ca.pem, the deployment anchor for certificates CDS issues. Keep it with your approved
pins and compare later CA reads before replacing it.
Keep the singleton running.
Its CA and secrets live in memory. Restarting it replaces that volatile state. Keep secret escrow and follow CDS lifecycle guidance before restarting.
5. Install and check the cluster components
c8s install --cvm-mode aks --hardware-platform sev-snp \
--image-tag '<IMAGE_TAG>' --operator-keys operator.pub \
--cds-address '<CDS_HOST>:8443' --cds-pin cds-bootstrap/cds-pin.jsonUse the same image tag as the seed command. The installer checks the configured component seed and node subnets, then runs Helm with readiness waiting enabled. Require successful completion before checking the resulting objects:
kubectl get pods -n c8s-system
kubectl get pods -n c8s-router
kubectl get mutatingwebhookconfiguration c8s-pod-injector \
-o jsonpath='{.webhooks[*].clientConfig.caBundle}'Expect component pods to be ready and the webhook CA bundle to be nonempty. The operator,
attestation API, and Node Resource Interface (NRI) image-policy installer/worker serve the
cluster; the router runs in c8s-router. CDS remains in its external VM. Every newly admitted pod in a
namespace matched by the webhook's namespaceSelector receives its own c8s-mesh endpoint,
with NRI enforcing nftables packet rules. See the
namespace coverage check
before selecting an application namespace.
On managed nodes, validate the containerd changes and privileged node components through node-image upgrades. If installation fails, inspect the pods and events, correct the cause, and rerun the same pinned install command. Readiness establishes health; the next check establishes the router's attested identity.
6. Verify the router from your workstation
Identify the node hosting a ready router pod:
kubectl get pods -n c8s-router -o wide
kubectl get nodes -o wideMatch the router pod's NODE column to the node list. Set <ROUTER_ADDRESS> in the next
command to that node's IP address reachable from your workstation. The default host port
serves the router on TCP 443. If only an internal address is listed, use the private-network
path prepared above. Stop and establish that path if the address is unreachable.
Pod port-forwarding is not a substitute: packet rules reject root-owned sockets entering a
pod's network namespace.
c8s verify 'https://<ROUTER_ADDRESS>:443' --kind lb \
--server-name c8s-router.c8s-router.svc \
--image-policy-file router-pin.json --mesh-ca mesh-ca.pemReplace <ROUTER_ADDRESS> with the reachable router address. --server-name supplies the
default certificate's DNS name even when connecting to an IP address. router-pin.json
checks the router node's approved identity; mesh-ca.pem checks its deployment's issuing
CA. The CDS image pin belongs to a different VM and cannot substitute for the router policy.
Require exit code 0 and inspect the verdict's warnings. Discovery binds the certificate to
issuance-time evidence, so expect fresh: false: it does not contain a nonce from this
verification attempt. The check also compares the live TLS serving certificate with the
attested certificate. Its result covers this connection to one router replica at one instant.
It does not verify an application's later connection or backend workload.
Exit 1 means invalid inputs, 2 means verification or policy failure, 3 means required
evidence or network material was unavailable, and 4 means partial verification. None is a
successful completion of this tutorial. See
Verify a deployment for verdict fields and fresh route
checks.
What you've built
You have a c8s installation whose components trust an external, launch-pinned CDS, a retained mesh CA, and a router checked against its own approved node policy. You have also checked Kubernetes readiness separately from attested identity.
Where to go next
Run a confidential workload adds an application, admits its complete container policy, publishes a signed route, and sends a request through a verified encrypted session. The application serves ordinary HTTP inside its pod; injected platform containers hold the private mesh credentials.
For the trust assumptions behind these steps, read trust boundaries and limitations.