Fetch a kubeconfig from a running CVM
Attest a c8s node CVM you launched yourself, prove it booted the image you pinned and was launched to trust your operator key, and write a 24-hour cluster-admin kubeconfig over RA-TLS.
c8s get-kubeconfig runs on your machine and gives you kubectl access to a c8s node
confidential VM (CVM) that no cluster you can reach knows about yet. It attests the guest through
the measured-image + operator-key gate, then exchanges a locally generated certificate signing
request (CSR) for a system:masters client certificate over Remote-Attestation TLS (RA-TLS) and
writes a kubeconfig. Its counterpart inside the guest is cred-release, a service in the c8s node
image that signs that certificate with the cluster's client CA.
Nothing is unsealed from storage. The credential is minted on demand, the private half of the client key never leaves your machine, and the certificate is valid for 24 hours — run the command again for a new one.
This is for node CVMs you launch yourself.
On AKS and GKE the provider issues the kubeconfig and this command does not apply. See Why this does not apply on AKS or GKE.
Before you start
- A booted c8s node CVM you can reach on
8400(attestation-api),8443(cred-release), and6443(kube-apiserver). See Boot a node CVM on bare metal. - That CVM must have been launched for your operator key. On Intel TDX the public key
reaches the guest as a disk labeled
opkeydataholding one file,pubkey, and the measured initrd extends its digest into RTMR[3] beforeswitch_root. On AMD SEV-SNP the launcher commitsHOSTDATA = SHA-256(pubkey bytes)at launch. Either way the binding is set at launch and cannot be added to a running guest. - The operator EC private key (
operator.key). - The build-artifact manifest of the exact image the CVM booted (step 1 gets it).
- On SEV-SNP, outbound HTTPS from your machine to AMD's Key Distribution Service: bare-metal SNP evidence carries no inline VCEK, so the verifier fetches one to check the quote's signature.
- The
c8sCLI — Install c8s has the build steps.
One keypair, two gates
One EC keypair does double duty across the whole bootstrap. Generate it once:
umask 077
openssl ecparam -name prime256v1 -genkey -noout -out operator.key
openssl ec -in operator.key -pubout -out operator.pub| Half | Goes to | Authorizes |
|---|---|---|
operator.pub | the CVM's launch identity (TDX RTMR[3], SNP HOSTDATA) and c8s install --operator-keys | which key the guest releases a credential to, and which keys the CDS accepts operator writes from |
operator.key | c8s get-kubeconfig --operator-key and c8s allowlist / c8s secrets writes | signing the release request, and signing allowlist and secrets writes |
These are two independent gates that in practice hold one keypair: the launch binding is verified
by the guest and by this command, while --operator-keys is a Helm value the install pins into
the CDS, the cluster's Certificate Distribution Service.
c8s get-kubeconfig derives the public half from --operator-key itself, so the bytes it
compares against the quote are the bytes openssl ec -pubout wrote.
That forces the order. The key must exist before the CVM boots, and the CVM has no kubeconfig until this command runs, so the bootstrap is:
- Create the keypair (above).
- Launch the node CVM with
operator.pubbound into its launch identity. - Run
c8s get-kubeconfig— this is the firstkubectlaccess anyone has to the guest. - Run
c8s install --operator-keys operator.pubagainst that kubeconfig.
Fetch the kubeconfig
The commands below assume a bare-metal Intel TDX guest running under KubeVirt, addressed by its
guest IP. Substitute <GUEST_IP>.
Get the image manifest for the build the CVM booted
The manifest ships as a manifest.json layer of the oras artifact published beside the CDI image
the CVM boots — the same repository and the same tag with the -cdi suffix removed. Pull the
layer:
crane manifest <REGISTRY>/<REPO>:<TAG> \
| jq -r '.layers[] | select(.annotations["org.opencontainers.image.title"] == "manifest.json") | .digest'
crane blob <REGISTRY>/<REPO>@<MANIFEST_LAYER_DIGEST> > image-manifest.json<TAG> is the boot image's tag without -cdi; <MANIFEST_LAYER_DIGEST> is the digest the first
command printed. Confirm it describes the build your CVM booted by matching the whole tuple,
not just mrtd — two builds that share a firmware and memory layout report the same MRTD while
their kernels, and so their RTMR[1], differ:
jq '.tdx' image-manifest.json # TDX: mrtd, rtmr1, rtmr2
jq '.snp_variants' image-manifest.json # SNP: one launch digest per vCPU countThe manifest's shape selects the platform: a TDX build publishes mrtd/rtmr1/rtmr2, an SNP
build publishes snp_variants. A generic artifact-hash manifest.json carries neither and is
rejected.
Confirm the guest is answering
curl -s http://<GUEST_IP>:8400/health # "platform":"tdx" (or "snp")
curl -sk -o /dev/null -w '%{http_code}\n' https://<GUEST_IP>:6443/livez # 200, 401, or 403
curl -sk -o /dev/null -w '%{http_code}\n' https://<GUEST_IP>:8443/ # any code but 000The attestation-api answers as soon as the TEE device is up. cred-release answers much later:
it starts after RKE2 has initialized and written the cluster client CA, which on a first boot
takes minutes.
Attest the CVM and fetch the kubeconfig
c8s get-kubeconfig \
--node <GUEST_IP> \
--operator-key operator.key \
--image-manifest image-manifest.json \
--out guest.kubeconfig--node fills in the three endpoints with their standard ports. On success the command writes
guest.kubeconfig with mode 0600 and prints:
wrote guest.kubeconfig (context "c8s") — attested: image tuple + operator-key chain verifiedWhile cred-release is still coming up you will see cred-release not listening yet; retrying
on stderr. The command retries a refused connection every 5 seconds for --release-wait
(2m by default); raise it on a slow first boot.
Add --workload-image <REF>@sha256:<HEX> once per digest-pinned image the node's measurer
extended into RTMR[3], in first-extend order. Omit it when the node runs no measured workloads —
the register must then equal the bare operator-key seed. The flag is TDX-only.
Confirm what you got
export KUBECONFIG=$PWD/guest.kubeconfig
kubectl auth whoami -o jsonpath='{.status.userInfo.username}' # operator
kubectl get nodesThe identity is operator in the group system:masters, and the kubeconfig anchors to the
cluster's serving CA, which the guest returned with the certificate. It expires 24 hours after
issue; re-run c8s get-kubeconfig for a fresh one. Next stop is
Install c8s, with --operator-keys operator.pub.
Address a KubeVirt guest by name
If the guest runs as a KubeVirt VirtualMachineInstance, name it instead of chasing its address:
c8s get-kubeconfig \
--vmi <NAMESPACE>/<VMI_NAME> \
--operator-key operator.key \
--image-manifest image-manifest.json \
--out guest.kubeconfig--vmi reads the VMI's first reported status.interfaces[].ipAddress through your current
kubeconfig — the outer cluster that hosts the VM — and then talks to the guest directly. The
namespace defaults to your current context's. A VMI that is still booting reports no address yet,
and the lookup fails with has no reported address. --vmi and --node are mutually exclusive.
SEV-SNP under direct QEMU
A guest launched by hand under QEMU usually reaches you on host-forwarded ports rather than
8400/8443/6443, so give all three URLs explicitly instead of --node:
c8s get-kubeconfig \
--attest-url http://127.0.0.1:<ATTEST_PORT>/attest \
--release-url https://127.0.0.1:<RELEASE_PORT> \
--apiserver-url https://127.0.0.1:<APISERVER_PORT> \
--operator-key operator.key \
--image-manifest manifest.json \
--out guest.kubeconfigUse the SNP build's manifest.json, the one carrying snp_variants. --workload-image is
rejected on SNP: there is no runtime-extend register, so a workload chain cannot be verified.
The kubeconfig's tls-server-name is c8s-cvm by default, which is the SAN the node image bakes
into RKE2's tls-san. That is what makes the apiserver certificate verify against a stable name
rather than the per-launch address you dialed.
What the gate checked
The manifest names the platform, and the node must be that platform: evidence tagged az-snp,
az-tdx, or gcp-* is refused before any claim is read. On top of that:
| Platform | The verified quote must carry |
|---|---|
| Intel TDX | MRTD, RTMR[1] (guest kernel), and RTMR[2] (guest rootfs) equal to the manifest's tuple, and RTMR[3] equal to the operator-key seed extended, in order, by each --workload-image digest |
| AMD SEV-SNP | a launch digest that is one of the manifest's per-vCPU-count variants, and a 32-byte HOSTDATA equal to SHA-256 of the operator public key |
The image tuple and RTMR[3] are both load-bearing. RTMR[3] alone is not an identity: the untrusted host stages the operator key, so it could boot any image at all and still reproduce the bare operator-key register. The manifest anchors the image; the register binds the key.
The same policy is enforced twice. Once on the attest gate, against a fresh 32-byte nonce
returned in report_data. Then again on the RA-TLS dial to cred-release, where the serving
certificate must be self-signed, carry a quote bound to its own public key, and satisfy the
identical pins — so a host that cannot forge the first gate cannot sit in the middle of the
channel that carries the CSR either. For how this verification model works across the rest of the
platform, see The verification model.
When it fails
Every check fails closed, and the message names which one.
| Message | Cause | Fix |
|---|---|---|
MRTD mismatch / RTMR[1] mismatch (guest kernel) / RTMR[2] mismatch (guest rootfs) | the manifest is from a different build than the one that booted | fetch the manifest published beside the image the CVM actually booted, and match the whole tuple |
RTMR[3] mismatch (operator-key + workload chain) | the guest was launched for a different key, or --workload-image is missing, extra, or out of order | confirm which operator.pub went on the opkeydata disk; list each measured image once, in first-extend order |
launch digest mismatch: … (a different guest image booted, or a vCPU count the manifest has no variant for) | SEV-SNP: the image or the vCPU count is outside the pinned set | launch at a vCPU count the manifest publishes a variant for |
HOSTDATA mismatch … (the VM was not launched for this key) or quote carries no HOSTDATA | SEV-SNP: the launcher committed another key's digest, or none | relaunch with host-data set to SHA-256(operator.pub) |
node platform is "az-snp" but --image-manifest pins "snp" | the target is a cloud CVM, not one you launched | use the provider's kubeconfig — see below |
image manifest …: missing "mrtd" or no "snp_variants" | a generic artifact-hash manifest.json, or the other platform's manifest | pull the manifest.json layer of the build's oras artifact |
cred-release not listening yet; retrying (stderr, not fatal) | boot ordering: RKE2 has not written the client CA yet | wait; raise --release-wait on a slow first boot |
credential release: release request: … EOF | the guest accepted the connection before its TLS listener was serving; only a refused connection is retried | run the command again |
release HTTP 401: unauthorized: … | the request was signed by a key the guest has not measured, or the two clocks differ by more than 60 seconds | check which key you passed; sync NTP on both ends |
every release dial refused for the full --release-wait | cred-release is skipped, not failed, on a guest with no opkeydata disk or on an agent-role node | relaunch the guest with the operator key, on a server-role node |
Why this does not apply on AKS or GKE
Each of these is enough on its own:
cred-releaseships only in the c8s node image. An AKS or GKE node does not run that image, so nothing serves:8443.- The gate refuses the evidence. A manifest pins
tdxorsnp; Azure and GCP confidential nodes reportaz-tdx,az-snp, orgcp-*, and the platform compare rejects them. - HOSTDATA is owned by the cloud stack on those platforms, so it cannot carry the operator-key binding at all.
Get the kubeconfig from the provider instead — az aks get-credentials on AKS. See
Provision a confidential cluster on Azure. The operator
keypair still matters there: c8s install --operator-keys operator.pub is what authorizes
allowlist and secrets writes.
See also
- Boot a node CVM on bare metal — the step that produces the guest this page attests.
- Install c8s — the next step, using the kubeconfig you just wrote.
- CLI reference — every flag, with its type and default.
- The verification model — how attested channels are checked elsewhere in c8s.
- Obtain launch measurements — the digest
c8s install --measurementswants.