Verified chat over confidential vLLM

1. Deploy a confidential vLLM backend

Run vLLM as a confidential workload. The confidential.ai/cw: vllm annotation opts the pod in — the value vllm is the workload id. The c8s webhook injects two get-cert containers, the c8s-cert identity sidecar and a c8s-cert-wait gate that holds your container until the first CDS-issued certificate lands, and the operator mints a headless Service c8s-vllm.<NAMESPACE>.svc. Headless DNS returns pod IPs, which the node mesh wraps in attested mTLS. We serve facebook/opt-125m, a tiny model that's quick to load on CPU. It defines no chat template (and transformers ≥ 4.44 supplies no default), so the manifest passes one inline — that's what lets Step 6's /v1/chat/completions answer.

# vllm.yaml
apiVersion: apps/v1
kind: Deployment
metadata: { name: vllm, namespace: workloads }
spec:
  replicas: 1
  selector: { matchLabels: { app: vllm } }
  template:
    metadata:
      labels: { app: vllm }
      annotations: { confidential.ai/cw: vllm }   # workload id → headless Service c8s-vllm
    spec:
      containers:
        - name: vllm
          image: vllm/vllm-openai-cpu:v0.22.1-x86_64   # official CPU image (see the callout above)
          args: ["--model", "facebook/opt-125m",   # tiny test model
                 "--chat-template", "{% for m in messages %}{{ m.role }}: {{ m.content }}\n{% endfor %}assistant:"]
          ports: [{ containerPort: 8000 }]         # vLLM's OpenAI server port
          volumeMounts:
            - { name: shm, mountPath: /dev/shm }   # vLLM needs more shm than the 64Mi default
      volumes:
        - name: shm
          emptyDir: { medium: Memory, sizeLimit: 1Gi }

kubectl create namespace workloads
kubectl apply -f vllm.yaml

You don't create a Service yourself — the operator mints the c8s-vllm headless Service from the annotation. (The pod won't actually start until its image is allowlisted, next.)

2. Allow the vLLM image

Nothing runs unless its image digest is on the allowlist — otherwise enforcement blocks the container at creation. Resolve the digest of the exact image you deployed:

crane digest vllm/vllm-openai-cpu:v0.22.1-x86_64
# sha256:…

Add it to the live allowlist with c8s allowlist, signed by the operator key you created when installing the cluster (operator.key — its public half is what you pinned with --operator-keys).

Where you point --url depends on your front door. The chart publishes the complete /allowlist API through router by default, so on a cluster whose front door still carries its CDS-issued certificate you use the same URL as application traffic. Not here: the front door you build in Step 3 terminates a real hostname with a WebPKI certificate, which isn't yet bound to the discovery attestation — so the CLI deliberately refuses it. Talk to the CDS directly instead. It has no public ingress, so port-forward it (stop Tutorial 1's LB port-forward first — both bind local port 8443):

kubectl port-forward -n c8s-system svc/c8s-cds 8443:8443 &

c8s allowlist add sha256:<VLLM_IMAGE_DIGEST> vllm/vllm-openai-cpu:v0.22.1-x86_64 \
  --url https://localhost:8443 \
  --measurements <CDS_LAUNCH_DIGEST> \
  --operator-key operator.key
# added sha256:<VLLM_IMAGE_DIGEST>

That's the only port-forward you need — the CLI verifies the CDS's RA-TLS attestation in-process, so nothing else has to be reachable. --measurements is the CDS launch digest; on a Node-as-CVM cluster that's the node measurement every pod on the node reports, which c8s verify prints for you. Omit it and you get:

warning: no --measurements set; accepting any attested endpoint build (UNSAFE)

Fine for a tutorial cluster, but pin it in production so a write can never land on a rogue CDS. The enforcement plugins poll the CDS-served list, so the vLLM pod from Step 1 starts on its next retry:

kubectl get pods -n workloads -w
# vllm-…   2/2   Running   ← the vLLM container plus the injected c8s-cert sidecar

3. Point the Load Balancer at vLLM

You don't hand-edit nginx. The chart wires the front door — router (public TLS + over-encryption) → your engine, over the node mesh's attested mTLS — and you aim it at your vLLM workload with --workload-ref + --upstream. The attestation and over-encryption endpoints your client needs are served by default, so there's nothing extra to switch on. Apply the routing with one upgrade — re-run your original c8s install with the two flags added:

c8s install \
  --single-node \
  --cvm-mode aks \
  --hardware-platform sev-snp \
  --operator-keys operator.pub \
  --measurements <M> \
  --workload-ref vllm=workloads/deployment/vllm:8000 \
  --upstream vllm

Repeat the flags you first installed with — --cvm-mode and --hardware-platform are required on every install, and --measurements <M> re-applies the mesh pin you closed Tutorial 1 with — and if you pass a -f values.yaml too, the values these flags derive win on the keys they set.

The ref's :8000 (vLLM's default port) derives router.upstream = c8s-vllm.workloads.svc.cluster.local:8000, so router dials vLLM's headless Service directly and that hop rides the node mesh's attested mTLS — hence no nginx surgery. The confidential.ai/cw: vllm stamp the ref applies is a no-op — Step 1's manifest already carries it. A request your client sends over the over-encryption tunnel (Step 6) is decrypted inside the LB enclave and forwarded over the mesh to vLLM's OpenAI-compatible /v1/chat/completions.

Giving that front door a public address is a separate, cluster-specific choice, and it's made in a values file rather than with flags: router.service.type: LoadBalancer for a public IP on AKS, router.san for the hostnames nginx answers on, and router.publicTLS.secretName for the certificate it presents. Pass the file with -f values.yaml alongside the flags above. The outer TLS layer is untrusted transport either way — Step 5's attestation is what establishes trust — but a WebPKI certificate is what lets a browser, or a stock Node client, reach https://chat.example.com without relaxing its own PKI check.

4. Gather the two pinned values

Verification is meaningless without pinning your cluster's identity out of band:

• LB launch measurement — the SHA-384 launch digest of the LB enclave. Use the published or recomputed digest (see Obtaining launch measurements). On a Node-as-CVM cluster this is the node's measurement, the same value the operator passed to c8s install --measurements — and the same one you pinned for the CDS in Step 2.
• Mesh CA certificate — your cluster's CDS mesh CA (PEM). Capture it from a trusted context at install time (the CDS publishes its public CA bundle at GET /ca). This is the cluster-unique anchor — why.

export LB_MEASUREMENT=<LB_LAUNCH_DIGEST>
# plus mesh-ca.pem, captured out of band from your cluster operator / CDS bootstrap

Both values come to you out of band on purpose. Reading them off the endpoint you are about to verify would prove nothing.

5. Verify and connect

Create a C8sClient with the LB origin and your pinned values, then connect(). That generates a nonce, fetches the LB attestation, verifies the attestation evidence in WASM, checks the measurement and the report_data binding, confirms the CDS cert chains to your mesh CA, and runs the X25519 + ML-KEM-768 handshake — all fail-closed:

import { C8sClient } from "c8s-verify";
import { readFileSync } from "node:fs";

const client = new C8sClient({
  baseUrl: "https://chat.example.com",            // your c8s LB
  platform: "az-snp",                             // Tutorial 1's cluster: Azure vTPM SEV-SNP
  measurements: [process.env.LB_MEASUREMENT],     // pinned LB launch digest
  meshCaPem: readFileSync("mesh-ca.pem", "utf8"), // pinned cluster anchor
});

const session = await client.connect();           // throws C8sVerifyError on any failure
console.error(`✓ verified LB enclave — measurement ${session.attestation.measurement}`);

If anything is off — wrong measurement, tampered evidence, or a cert that doesn't chain to your mesh CA — connect() throws and no channel is opened. See what it verifies.

6. Chat over the sealed channel

session.fetch seals the entire request and sends it down the tunnel; the LB enclave decrypts it, forwards it to vLLM over the mesh, and seals the response back. Call vLLM's OpenAI-compatible endpoint — the model must match what you served in Step 1:

async function ask(session, messages) {
  const res = await session.fetch("/v1/chat/completions", {
    method: "POST",
    headers: { "content-type": "application/json" },
    body: JSON.stringify({ model: "facebook/opt-125m", messages }),
  });
  if (res.status !== 200) throw new Error(`vLLM returned HTTP ${res.status}: ${res.text()}`);
  return JSON.parse(res.text()).choices[0].message.content;
}

Wrap it in a REPL for an actual chat loop (chat.mjs):

import { createInterface } from "node:readline/promises";
import { stdin as input, stdout as output } from "node:process";

const rl = createInterface({ input, output });
const history = [];
for (;;) {
  const prompt = await rl.question("you> ");
  if (!prompt || prompt === "/exit") break;
  history.push({ role: "user", content: prompt });
  const reply = await ask(session, history);       // session from Step 5
  history.push({ role: "assistant", content: reply });
  console.log(`bot> ${reply}\n`);
}
rl.close();

Run it:

LB_MEASUREMENT=$LB_MEASUREMENT node chat.mjs

Keep "stream": true out of these requests. The tunnel seals one whole request envelope and one whole response envelope, so a token stream can't be delivered incrementally through it. Streaming does work through router's ordinary route — nginx forwards Server-Sent Events straight through rather than buffering them — but that path gives you TLS to the front door, not a channel sealed to the enclave. Pick per endpoint: sealed for the prompts that matter, streamed for the ones where latency does. That ordinary route is the front door's default for every client: sealing is a per-client opt-in, and a stock HTTPS client that never calls connect() just gets plain TLS to the LB.