Verified chat over confidential vLLM

1. Deploy a confidential vLLM backend

Run vLLM as a confidential workload. The confidential.ai/cw: vllm annotation opts the pod in — the value vllm is the workload id — so the operator injects the get-cert identity (and, under --kata, the confidential RuntimeClass) and mints a headless Service c8s-vllm.<namespace>.svc. Headless DNS returns pod IPs, which the node mesh wraps in attested mTLS. We serve facebook/opt-125m, a tiny model that's quick to load on CPU.

# vllm.yaml
apiVersion: apps/v1
kind: Deployment
metadata: { name: vllm, namespace: workloads }
spec:
  replicas: 1
  selector: { matchLabels: { app: vllm } }
  template:
    metadata:
      labels: { app: vllm }
      annotations: { confidential.ai/cw: vllm }   # workload id → headless Service c8s-vllm
    spec:
      containers:
        - name: vllm
          image: <your-registry>/vllm-cpu:v0.6.3   # your CPU build (see the callout above)
          args: ["--model", "facebook/opt-125m"]   # tiny test model
          ports: [{ containerPort: 8000 }]         # vLLM's OpenAI server port

kubectl create namespace workloads
kubectl apply -f vllm.yaml

You don't create a Service yourself — the operator mints the c8s-vllm headless Service from the annotation. (The pod won't actually start until its image is allowlisted, next.)

2. Allow the vLLM image

Nothing runs unless its image digest is on the allowlist — otherwise enforcement blocks the container at creation. Resolve the digest of the exact image you deployed:

crane digest <your-registry>/vllm-cpu:v0.6.3
# sha256:…

Add it to the live allowlist with c8s allowlist, signed by the operator key you created when installing the cluster (operator.key — its public half is what you pinned with --operator-keys). The CDS has no public ingress, so port-forward it first — along with the attestation-api, which the CLI uses to verify the CDS's own attestation before talking to it:

kubectl port-forward -n c8s-system svc/c8s-cds 8443:8443 &
kubectl port-forward -n c8s-system svc/c8s-attestation-api 8400:8400 &

c8s allowlist add sha256:<vllm-image-digest> <your-registry>/vllm-cpu:v0.6.3 \
  --url https://localhost:8443 \
  --attestation-api-url http://localhost:8400 \
  --operator-key operator.key
# added sha256:<vllm-image-digest>

The CLI will warn that no --measurements are pinned — it's accepting any attested CDS, which is fine for this tutorial cluster; in production pin your CDS launch digest so a write can never land on a rogue CDS. The enforcement plugins poll the CDS-served list, so the vLLM pod from Step 1 starts on its next retry:

kubectl get pods -n workloads -w
# vllm-…   2/2   Running   ← the vLLM container plus the injected c8s-cert sidecar

3. Point the Load Balancer at vLLM

You don't hand-edit nginx. The chart wires the front door — tls-lb (public TLS + over-encryption) → your engine, over the node mesh's attested mTLS — and you aim it at your vLLM workload with --workload-ref + --upstream, plus a small values file for the attestation endpoints:

# values.yaml
tlsLb:
  attest:
    enabled: true     # serve the c8s-verify/v1 attestation + over-encryption endpoints

Apply the routing with one upgrade — re-run your original c8s install with the two flags and -f values.yaml added:

c8s install \
  --workload-ref vllm=workloads/deployment/vllm:8000 \
  --upstream vllm \
  -f values.yaml   # plus the flags you first installed with (e.g. --single-node --operator-keys operator.pub)

The ref's :8000 (vLLM's default port) derives tlsLb.upstream = c8s-vllm.workloads.svc.cluster.local:8000, so tls-lb dials vLLM's headless Service directly and that hop rides the node mesh's attested mTLS — hence no nginx surgery. The confidential.ai/cw: vllm stamp the ref applies is a no-op — Step 1's manifest already carries it. A request your client sends over the over-encryption tunnel (Step 6) is decrypted inside the LB enclave and forwarded over the mesh to vLLM's OpenAI-compatible /v1/chat/completions.

4. Gather the two pinned values

Verification is meaningless without pinning your cluster's identity out of band:

• LB launch measurement — the SHA-384 launch digest of the LB enclave. Use the published or recomputed digest (see Obtaining launch measurements).
• Mesh CA certificate — your cluster's CDS mesh CA (PEM). Capture it from a trusted context at install time (the CDS publishes its public CA bundle, e.g. GET /ca). This is the cluster-unique anchor — why.

export LB_MEASUREMENT=<sha384-launch-digest-of-the-lb>
# plus mesh-ca.pem, captured out of band from your cluster operator / CDS bootstrap

5. Verify and connect

Create a C8sClient with the LB origin and your pinned values, then connect(). That generates a nonce, fetches the LB attestation, verifies the attestation evidence in WASM, checks the measurement and the report_data binding, confirms the CDS cert chains to your mesh CA, and runs the X25519 + ML-KEM-768 handshake — all fail-closed:

import { C8sClient } from "c8s-verify";
import { readFileSync } from "node:fs";

const client = new C8sClient({
  baseUrl: "https://chat.example.com",            // your c8s LB
  measurements: [process.env.LB_MEASUREMENT],     // pinned LB launch digest
  meshCaPem: readFileSync("mesh-ca.pem", "utf8"), // pinned cluster anchor
});

const session = await client.connect();           // throws C8sVerifyError on any failure
console.error(`✓ verified LB enclave — measurement ${session.attestation.measurement}`);

If anything is off — wrong measurement, tampered evidence, or a cert that doesn't chain to your mesh CA — connect() throws and no channel is opened. See what it verifies.

6. Chat over the sealed channel

session.fetch seals the entire request and sends it down the tunnel; the LB enclave decrypts it, forwards it to vLLM over the mesh, and seals the response back. Call vLLM's OpenAI-compatible endpoint — the model must match what you served in Step 1:

async function ask(session, messages) {
  const res = await session.fetch("/v1/chat/completions", {
    method: "POST",
    headers: { "content-type": "application/json" },
    body: JSON.stringify({ model: "facebook/opt-125m", messages }),
  });
  if (res.status !== 200) throw new Error(`vLLM returned HTTP ${res.status}: ${res.text()}`);
  return JSON.parse(res.text()).choices[0].message.content;
}

Wrap it in a REPL for an actual chat loop (chat.mjs):

import { createInterface } from "node:readline/promises";
import { stdin as input, stdout as output } from "node:process";

const rl = createInterface({ input, output });
const history = [];
for (;;) {
  const prompt = await rl.question("you> ");
  if (!prompt || prompt === "/exit") break;
  history.push({ role: "user", content: prompt });
  const reply = await ask(session, history);       // session from Step 5
  history.push({ role: "assistant", content: reply });
  console.log(`bot> ${reply}\n`);
}
rl.close();

Run it:

LB_MEASUREMENT=$LB_MEASUREMENT node chat.mjs