The trust root

How measured CDS authority, volatile CA state, attestation, and sandbox inventory establish pod mesh identity.

The Certificate Distribution Service (CDS) runs in an external confidential VM (CVM). Its trusted execution environment (TEE) protects the mesh certificate authority (CA) and secret state from the host. CDS verifies node attestation evidence, checks sandbox inventory, and signs the certificates that authenticate pods to the mesh.

Measured authority

A launch measurement is a cryptographic digest representing the VM's launch state. A pin supplies independently expected measurements and configuration values for a verifier to compare with hardware evidence.

CDS starts with a required c8s-cds/v1 configuration file. Before serving, it verifies evidence from its own VM and checks that the exact file bytes match the launch configuration that evidence binds. The file contains public inputs: cluster identity, operator public keys, initial workload policy, node-image reference values, inventory address ranges (nodeCIDRs), DNS certificate naming rules, and optional high-availability (HA) settings. The host can read it, so private keys and secret values belong outside this document.

The configuration must name at least one operator key and one node subnet; default routes are rejected as node ranges. An empty node-image policy refuses certificate issuance. These are separate trust inputs:

Trust inputWhat it establishes
CDS identity pinThe expected CDS VM image and exact launch configuration
CDS node policyWhich node images CDS accepts for issuance and inventory callbacks
Verifier's node or router policyThe target image and launch configuration the verifier independently expects
Mesh CA anchorWhich issuer the verifier trusts for sandbox and workload claims

The measured configuration establishes operator authority and the singleton's node policy. For HA, it supplies the default node set for the operator-signed Genesis command that starts the replica group. Genesis selects HA's initial node policy and can specify another set with --nodes; later signed commands can change it. Authorized allowlist writes change workload policy. The CDS identity pin therefore does not establish which node policy is currently enforced. c8s cds verify --served-policy-file compares the served set with an independently intended current policy. See Verify a deployment.

Mesh CA bootstrap

The mesh CA signs with ECDSA, an elliptic-curve signature algorithm, using a P-384 key. Its default common name is c8s Mesh CA, and its certificate validity is 8760h (one year). Singleton settings --ca-common-name and --ca-cert-validity control those values; HA initializes its shared CA with the defaults. The public certificate is available at /ca through the verified CDS endpoint.

The measured configuration selects the state lifecycle. Quorum means a majority of the configured voting seats:

Mode or eventCA and state behavior
Singleton, with no ha blockEach process start generates a fresh CA key; secrets and sandbox bindings are held in that process's memory. Allowlist, route, and publication-history state follow the database's lifetime.
Replicated CDS, with an ha blockAn explicit operator-signed Genesis command starts the replica group. Its volatile replicated state includes the CA private key, secrets, allowlist, routes, and sandbox bindings.
One replica restartsIt can obtain state from surviving peers when the cluster's admission and serving requirements are satisfied. A replica restart alone does not imply a new CA.
Quorum is lostOrdinary APIs stop serving when the leader-confirmed lease ceases to hold; surviving memory remains.
All replica memory is lostThe volatile CA and secret state are lost; replication supplies no durable recovery from that event.

A serving replica needs a valid, time-bounded lease confirmed by its leader and the corresponding state applied locally. Preserve surviving VMs during a halt to retain their state. Preserving that state does not itself restore service: after permanent quorum loss, the current runtime has no supported workflow for the surviving minority to resume serving. Submitting a signed Recover command to a replica does not activate recovery. See the HA API contract.

CA certificate renewal retains the signing key. Injected pods support that renewal and bind their issuer key for their lifetime. Replacing the CA key, including through HA rotate-key, requires pod recreation to adopt the new issuer. New mesh connections between different CA keys fail authentication; key replacement does not promise immediate termination of existing streams. Operator authorization keys are a separate role: changing them requires a new measured configuration and its pin.

The attestation flow

The pod's get-cert authenticates CDS using remote attestation TLS (RA-TLS), a TLS channel whose certificate carries hardware evidence. It creates a certificate signing request (CSR) and obtains a 32-byte challenge from /authenticate. The challenge binds both the node evidence and the sandbox token to this CSR key.

Requesting node TEE                         Separate CDS TEE
+------------------------------------+     +----------------------------------+
| get-cert in the pod                 |     | CDS, with measured launch config |
| node attestation API               |     | CDS-local attestation API        |
| node sandbox inventory             |     | mesh CA and live policy          |
+------------------------------------+     +----------------------------------+
    get-cert -- POST /authenticate ------------> CDS
    get-cert <----------- challenge ------------ CDS
    get-cert --> local inventory: token bound to key + challenge
    get-cert --> node attestation API: evidence bound to key + challenge
    get-cert -- POST /attest ------------------> CDS
                CSR + evidence + sandbox token   |
    inventory <---- GET /identity over RA-TLS ----+
    inventory ----- signing key ---------------->+ verify sandbox token bindings
                                                 +--> CDS-local API: verify evidence
                                                 |    and key/challenge binding
                                                 +--> node-image and CSR policy
    inventory <---- GET /digests/<sandboxID> -----+    over RA-TLS
    inventory ----- admitted containers -------->+ allowlist and workload matching
    get-cert <----- leaf + mesh CA ---------------+ sign certificate

For /attest, CDS consumes the challenge, verifies the CSR's self-signature, and checks the sandbox token if present. The constrained inventory signing-key lookup happens before CDS verifies the requesting node's hardware evidence. Both inventory callbacks use mutually attested TLS.

The hardware evidence's report_data field carries additional data supplied by the requester. CDS requires it to contain SHA-384(CSR public key ‖ challenge), where ‖ means concatenating the encoded public-key bytes and the challenge bytes. A valid evidence signature authenticates this key/challenge binding. CDS separately matches the evidence against one complete entry in its current node-image policy, including that entry's expected values for the platform's measurement registers.

CDS validates the requested subject alternative names (SANs) and common name against its certificate policy. For a sandbox-bound request, it checks the inventory's admitted image digests against one allowlist snapshot, uses that same snapshot for workload matching, and signs the leaf. Missing inventory evidence or a denied image refuses that request. The general API also accepts requests without a sandbox token; those leaves lack sandbox identity and cannot authenticate to the pod mesh.

Challenges expire after --challenge-ttl (default 60s). A singleton consumes a challenge once. HA replicas can accept a challenge minted by another serving replica, but each tracks consumption locally: the guarantee is once per replica. The evidence and token remain bound to the same CSR key and challenge. See the issuance API for the wire contract.

What an issued leaf carries

A leaf certificate is the pod's certificate, signed by the mesh CA. Its lifetime is set by --cert-ttl and capped at 24 hours. A leaf with a matched-workload stamp is additionally capped by --named-cert-ttl, whose default and ceiling are six hours.

An object identifier (OID) names an X.509 certificate extension:

OIDExtensionPresence and contents
1.3.6.1.4.1.66378.1.1RA-TLS attestationOptional evidence copied verbatim from the CSR when supplied
1.3.6.1.4.1.66378.1.2Audit digestSHA-256 of the serialized evidence CDS accepted for this issuance; present on every issued leaf
1.3.6.1.4.1.66378.1.4Pod sandbox IDPresent when CDS verified a sandbox token
1.3.6.1.4.1.66378.1.5Matched workloadPresent when the sandbox's complete admitted workload container set uniquely matches an allowlist entry; contains its name and the policy's version and canonical digest

The .1.1 evidence that get-cert places in its CSR binds the bare public key, without the challenge. It is separate from the challenge-bound evidence CDS checks for issuance. CDS copies this optional extension without re-verifying it; a relying party that uses it must verify its evidence and key binding itself. The .1.2 digest records which issuance evidence was accepted and cannot replace that evidence.

The sandbox and workload stamps are authenticated by the CA signature. c8s verify --sandbox-id, --workload, and --allowlist require --mesh-ca to check them. A workload stamp records the issuance-time match and can remain usable until certificate replacement or expiry. get-cert --expected-workload keeps renewal polling active while the stamp has another name or is absent; it does not make a mismatch an issuance failure.

Sandbox identity

All pods on a confidential node share its hardware measurement. The pod sandbox is the container runtime's pod instance; its ID distinguishes the instance behind a certificate key. The inventory, supplied by the nri-image-policy admission plugin, derives that identity from the caller and reports the containers admitted to that sandbox.

Its local token route and network evidence routes have separate roles:

  • get-cert sends its public key and CDS challenge to POST /sandbox over a node-local Unix socket. Kernel peer credentials (SO_PEERCRED and SO_PEERPIDFD) identify the calling process, which the inventory resolves through its kernel control group (cgroup) and container record to a sandbox. Workload claims are enabled by default; injected clients never set the standalone --no-workload-claims opt-out and fail closed when their inventory is unavailable.
  • The inventory signs a token containing the sandbox ID, SHA-256 of the requester public key, challenge, and inventory IP. get-cert forwards it to CDS. The token supplies no trusted signing-key credential.
  • CDS reads the unverified inventory IP only to select a constrained dial target. It must be a unicast IP literal within the measured configuration's nodeCIDRs. CDS authenticates the endpoint over mutually attested TLS at port 1019, obtains its key from /identity, and verifies the token signature, challenge, and requester-key binding.
  • CDS obtains /digests/<sandboxID> from that inventory. The response includes admitted image digests and container arguments, environment, and mount evidence. An empty digest inventory refuses issuance.

The address ranges must distinguish node inventory addresses from tenant pod addresses. The fixed privileged port 1019 is one part of the boundary: tenant pods must also be prevented from publishing a responder on a node address through hostNetwork or hostPort. The chart's deny-host-namespaces policy supplies these admission restrictions. The self-managed bare-metal mode uses a baked node image with a measured runtime wrapper. In its locked build, the wrapper rejects container exec operations, including kubectl exec, and restoring containers from runtime checkpoints; it also enforces a minimum container privilege policy. Chart-managed enforcement on GKE and AKS depends on cluster administrator control. These protections must hold in the admitted node image and deployment mode; a shared node measurement alone cannot identify the inventory process.

What vouches for a sandbox ID — and what does not

The sandbox ID is in the leaf's signed area, outside the hardware evidence's REPORTDATA. The CA signature vouches for it. Without a trusted mesh CA, a reported sandbox ID is an unauthenticated claim.

Issuance requires allowlist membership for every admitted image. The workload stamp requires the complete admitted workload container set to match exactly one allowlist entry. Every reported workload container must satisfy that entry's image, argument, environment, and mount policies, and every declared main container must be represented. Injected mesh and credential containers are excluded from this workload match while their images remain subject to the membership check. During startup, only part of a workload may have been admitted; that pod can receive a sandbox-bound certificate before it earns a workload stamp. The inventory retains the admission record of completed and removed containers, including init containers. Their completion does not erase them from workload matching. Admission and ongoing enforcement are described in The allowlist.

The inventory remains part of the node's trusted computing base. A privileged actor able to answer as that inventory on the node's address can claim sandbox identities. See Node trust boundary.

Private pod mesh credentials

The webhook injects the certificate client and mesh endpoint into each covered pod. get-cert publishes the validated leaf, private key, and CA as one credential generation on the private volume at /etc/c8s/certs. Platform consumers adopt a coherent generation. The webhook rejects nonempty credential path or volume overrides and application mounts of that volume. Applications serve ordinary TCP or HTTP inside the pod while the injected endpoint handles mesh TLS.

Mesh peers require TLS 1.3, a chain to their mesh CA, valid certificate times and purpose, the c8s-mesh/1 application protocol, and a sandbox ID. The ordinary pod mesh performs no per-connection hardware-evidence verification or destination workload-name authorization. Consumers that require a particular workload must check the CA-authenticated workload stamp. When credentials expire or are withdrawn, the endpoint becomes unready and refuses new forwarding; this does not establish that existing streams immediately close. See The verification model.

The image allowlist

CDS serves the workload policy at /allowlist and retains publication history at /allowlist/history. The issuance gate uses the same policy store. The measured configuration names the operator public keys authorized to change it; their private counterparts remain with the operator.

For an allowlist write, the CLI signs a short-lived token bound to the HTTP method, path, and exact body. CDS verifies those bindings and spends the token before attempting the policy update. GET /operator-keys exposes the public authorization set, and c8s cds verify --operator-keys can compare it with an independently expected bundle. See The allowlist for policy semantics and Manage the allowlist for the write workflow.

Verifying the CDS itself (RA-TLS)

CDS uses its own attested serving certificate for RA-TLS, independently of the mesh CA certificates it issues. Its platform comes from the verified self-report at startup.

CDS clients check the serving certificate's evidence, public-key binding, and expected image and exact configuration before trusting responses. Injected credential clients use the CDS endpoint and pins supplied by their node. The CDS pin is independent of the requesting node's policy. For an operator check, c8s cds verify --image-policy-file <cds-pin.json> pins CDS's identity; --served-policy-file separately checks the current admitted-node set. See Verify the CDS.

The external CDS endpoint listens on port 8443 by default. The router proxies the allowlist API; other APIs, including /ca, /secrets, and /routes, use the verified CDS endpoint directly. A public CA fetched from that verified endpoint can anchor mesh claims.

Browser clients use the router's attested session proof, which binds its CDS-issued leaf and issuing CA. The consumer still needs an independently trusted CA anchor to establish the intended cluster. See consumer verification.

See also