The verification model
Who verifies what in a c8s cluster — services verifying each other automatically over RA-TLS, an operator verifying a component by hand, and an end consumer verifying the front door from a browser it cannot extend — plus the over-encryption design that makes the last one possible.
Attestation is only as good as somebody's ability to check it. In a c8s cluster three parties do, and they reach the same enclave by different routes:
| Who | When | How |
|---|---|---|
| Services, on each other | continuously, on every connection | RA-TLS — evidence in the peer's serving certificate, checked mid-handshake |
| The operator, on a component | after install, then on a schedule from CI | c8s verify — dial it, pull its evidence, check it against a pinned measurement |
| The end consumer, on the front door | per session, from a browser | over-encryption — a second, attested channel inside the ordinary HTTPS one |
The first two both pull evidence out of an X.509 extension mid-handshake. Browsers expose no API to do that, so the consumer path cannot work the same way — and the consumer is the party whose independent check matters most. The design that works anyway is the bulk of this page.
Service-to-service verification (RA-TLS)
Inside the cluster you rarely verify by hand. Backend components authenticate each other with
RA-TLS: a workload's get-cert and
the ratls-mesh proxies pull the peer's attestation evidence out of its serving certificate during
the handshake, check it against the mesh CA, and only then complete the connection. That
verification is automatic and continuous — there is no operator step.
Two things a CDS-issued leaf now carries, which change what a relying party can re-check:
- The evidence CDS accepted at issuance, stamped onto the leaf. A peer doing RA-TLS-mode verification sees exactly what CDS verified instead of falling back to the CA chain alone.
- The pod sandbox ID, when the requester presented an inventory-signed sandbox token. It sits
in the leaf's signed area, not in REPORTDATA — the mesh CA signature is what vouches for it,
which is why
--sandbox-idbelow requires--mesh-ca.
The mesh's RA-TLS verifier accepts the snp, gcp-snp, az-snp, tdx, and az-tdx evidence
shapes. az-tdx is an Azure confidential VM's TD quote wrapped in the vTPM HCL report — the
Azure TDX path has no /dev/tdx-guest — and pins MRTD as the launch measurement.
What you verify by hand is the trust root: confirm the CDS itself is a genuine TEE at your
expected measurement with c8s cds verify.
Everything else chains to it.
The verifier engine
c8s verify does not embed a verifier of its own. It verifies in-process with
attestation-go — the Go port of the same attestation-rs engine the cluster runs — so there's
no container to launch and no service to reach. Using the cluster's own engine means the
product line never has to be supplied by hand: it auto-detects the platform and AMD product,
including Zen4c (Siena / Bergamo), which stock go-sev-guest cannot classify. Any AMD
collateral that isn't shipped inline (e.g. the VCEK for a bare report) is fetched from AMD
KDS. Intel TDX quotes verify against an Intel SGX Root CA pinned inside the verifier and
fingerprint-checked when it loads — no Intel endpoint is contacted.
The vendor signature chain, the REPORTDATA binding, the debug-guest rejection, and the
minimum-TCB floor are enforced inside the verifier. The launch-measurement allowlist
(--measurements) has no verifier-side input, so it is enforced client-side and fails
closed: if the report's launch digest isn't in your allowlist, verification fails.
What the front door exposes
The consumer-facing surface is the c8s Load Balancer (tls-lb): an nginx front end plus a
cds-attest sidecar, both inside the TEE (its own CVM under Pod-as-CVM, the node's under
Node-as-CVM). The sidecar ships enabled by default (tlsLb.attest.enabled), so a stock install
already answers browser verifiers; install with --attest=false to omit it.
| Path | Served by | Carries |
|---|---|---|
/.well-known/c8s/attestation | cds-attest | nonce-bound evidence, session keys, mesh leaf + CA, identity proof |
/.well-known/c8s/handshake | cds-attest | the hybrid key-agreement exchange |
/.well-known/c8s/tunnel | cds-attest | over-encrypted application records |
/v1/discovery | nginx | the LB's issued cert + evidence with the VCEK inline |
/allowlist, /allowlist/ | nginx → in-pod proxy | CDS's image allowlist API |
/.well-known/cds-cert.pem, /.well-known/mesh-ca.pem | nginx | static PEM files for in-cluster discovery |
All of those locations answer any origin by default. tlsLb.cors.protocolEndpoints (default
true) puts Access-Control-Allow-Origin: * on them, with no
Access-Control-Allow-Credentials, and short-circuits preflight OPTIONS at nginx with a 204
advertising GET, POST, OPTIONS, the headers Authorization, Content-Type, X-C8s-Session, and a
600-second cache. That is deliberate:
these endpoints exist so any browser anywhere can verify the cluster, every response is either
self-authenticating (hardware evidence, CDS-signed certificates, sealed tunnel records) or public
by design, and no request on them relies on ambient browser credentials — allowlist writes are
authorized by an operator-signed token over method, path, and body, not by a cookie.
Two boundaries to keep straight:
- Workload traffic is not covered.
tlsLb.routesand the catch-all upstream get no CORS headers from this default. Setting the operator's owntlsLb.cors.enabled: trueblock replaces the built-in policy everywhere, protocol endpoints included;tlsLb.cors.protocolEndpoints: falseremoves CORS from the protocol endpoints entirely. - Reading from this front door is not verification.
/.well-known/mesh-ca.pemand a plainGET /allowlistboth arrive over ordinary TLS through whatever terminator sits in front of the LB. They are discovery material, not trust anchors — pin the mesh CA out of band (below), and see Image allowlist for the attested way to read the allowlist.
The idea: over-encryption, not certificate extensions
Instead of hiding attestation inside the TLS certificate, the LB serves it over a normal HTTPS endpoint, via a challenge/response, and the client establishes a second, attested channel inside the existing connection:
┌─────────────────┐ ┌─────────────────┐ ╔══════════════════════╗
│ Browser │ │ TLS terminator │ ║ LB inside a TEE ║
│ c8s-verify │ │ untrusted │ ║ nginx + cds-attest ║
└────────┬────────┘ └────────┬────────┘ ╚══════════┬═══════════╝
│ │ │
│ GET .../attestation?nonce=N (N = 32 bytes)│
├─────────────────────┼──────────────────────►│
│ │ ┌────────────────┴───────────┐
│ │ │ mint per-session X25519 + │
│ │ │ ML-KEM-768 key │
│ │ │ transcript = H(version, │
│ │ │ mesh CA, mesh leaf, │
│ │ │ both pubkeys, N) │
│ │ │ HW report_data = transcript│
│ │ │ mesh leaf signs transcript │
│ │ └────────────────┬───────────┘
│ evidence + pubkeys + leaf/CA PEM + proof │
│◄────────────────────┼───────────────────────┤
┌────────┴─────────────────┐ │ │
│ verify in WASM: │ │ │
│ hardware chain │ │ │
│ measurement ∈ allowlist │ │ │
│ report_data = transcript│ │ │
│ leaf → pinned mesh CA │ │ │
│ proof signature │ │ │
└────────┬─────────────────┘ │ │
│ POST .../handshake (client X25519 + ML-KEM ciphertext)
├─────────────────────┼──────────────────────►│
│◄────────────────────┼───────────────────────┤ session_id
│ AES-256-GCM = HKDF(secrets, salt=transcript)
│ POST .../tunnel + X-C8s-Session (sealed) │
├─────────────────────┼──────────────────────►│
│◄────────────────────┼───────────────────────┤ sealed response
▼ ▼ ▼The plain HTTPS hop is never trusted; every check runs on the returned payload, and the channel terminates inside the LB's enclave.
A malicious TLS-terminating proxy may sit in front of the real LB. Verification is performed entirely on the returned payload, and only then does the client derive an end-to-end encrypted channel to the attested per-session key. Everything after that is confidential to the LB's enclave regardless of the outer TLS terminator.
The post-quantum over-encryption channel
The channel uses a hybrid KEM — X25519 (classical, WebCrypto) + ML-KEM-768
(post-quantum, mlkem-wasm) — following the TLS X25519MLKEM768 convention:
- Encapsulate against the attested ML-KEM key →
(mlkem_ct, mlkem_ss). - ECDH an ephemeral X25519 key with the attested X25519 key →
x25519_ss. ikm = mlkem_ss ‖ x25519_ss.key = HKDF-SHA256(ikm, salt = transcript, info = "c8s-verify/v1/over-encryption", L = 32)→ AES-256-GCM.
The salt is the identity transcript the hardware report committed to, not the bare nonce, so the derived key is worthless to anyone who could not produce that report.
Application traffic then rides a single POST /.well-known/c8s/tunnel endpoint as CBOR, one
AES-256-GCM record per message with a fresh 12-byte IV, keyed to a session the client names with
the X-C8s-Session header. The entire request is sealed — method, path, headers, and body —
so a proxy in front of the LB sees only ciphertext, not even the path or Authorization header.
The LB enclave opens the record, forwards the plaintext request to the backend over the cluster's
RA-TLS mesh, and seals the response back.
Three limits worth knowing:
- Replay is rejected. Each channel remembers the IV of every record it has successfully opened and refuses a repeat, so the untrusted terminator cannot resubmit a captured request for a second authenticated backend action. The set is bounded (4096 records); past that the channel fails closed and must be re-established.
- Sessions expire.
--session-ttl(default 5 minutes) bounds both the pending handshake between the attestation fetch and thePOST /handshake, and the idle life of an established session. - Records are size-capped. A sealed request is read up to 8 MiB; an upstream response larger than 32 MiB is refused rather than sealed.
Streaming responses
The tunnel is request/response: cds-attest reads the whole backend response, seals it as one
record, and returns it. A Server-Sent Events body from a streaming inference backend therefore
arrives in a single chunk at the end, and is bounded by the 32 MiB response cap.
Streaming works on the plain front-door path — the nginx catch-all upstream, without
over-encryption. That location sets proxy_buffering off and proxy_http_version 1.1, so tokens
are forwarded as the backend emits them instead of being accumulated and released in clumps. The
two are a real trade: over-encryption to the enclave, or token-by-token streaming.
Why cluster identity is pinned
The client pins two things out of band: the LB measurement allowlist and the mesh CA certificate. Pinning a measurement alone is not enough, and the reason is cluster identity:
The CDS and LB images are open source and reproducible — which is what makes them auditable — but it also means a valid measurement only proves "a genuine instance of the audited code on real silicon," not "my cluster." An attacker could stand up their own genuine LB enclave (same image, valid measurement, real vendor chain) and proxy you to it. The one value that is unique per cluster is the mesh CA key, generated inside the CDS TEE — so the client must pin something cluster-unique, and today that is the mesh CA certificate.
The binding closes the copy attack: report_data commits the exact mesh leaf and its issuing
CA, and the LB signs the same transcript with the leaf's private key. Copying a victim
cluster's public certificate chain gets an attacker nothing without that private key, which never
leaves the CDS-issued pod's TEE. There is no legacy or downgrade binding — the attestation
endpoint rejects a binding parameter outright, and rejects any nonce that is not exactly 32
bytes.
The proof signature is ECDSA, so cluster authentication is classical. X25519 + ML-KEM-768 gives the session key hybrid confidentiality; this path does not claim post-quantum authentication.
Transitivity of trust
The consumer only verifies the LB. Because the LB and CDS are open-source c8s components whose code anyone can audit, and because the LB's leaf chains to the CDS mesh CA — as do all other pods in the cluster — relying on the LB as a single point of trust transitively vouches for the backend pods it talks to over RA-TLS. The c8s mesh carries the rest.
See also
- c8s-verify (JavaScript) — the library that implements the consumer path.
- Verify a deployment — the operator path, command by command.
- The trust root — what the CDS vouches for, and how it is itself verified.