Verified chat over confidential vLLM

1. Deploy a confidential vLLM backend

Set the image reference and external CDS URL, replacing all angle-bracket placeholders:

export VLLM_IMAGE='<REGISTRY>/<REPOSITORY>@sha256:<IMAGE_DIGEST>'
export VLLM_READY_PATH='<IMAGE_READINESS_PATH>'
export CDS_URL='https://<CDS_HOST>:8443'
export CHAT_HOST='c8s-router.c8s-router.svc'

<IMAGE_DIGEST> is the validated image's 64-hex manifest digest. <IMAGE_READINESS_PATH> is the producer-confirmed readiness path from the image contract above. <CDS_HOST> is the VM endpoint from installation. CHAT_HOST matches the default router certificate's DNS name.

Save the deployment using that image:

cat > vllm.yaml <<EOF_MANIFEST
apiVersion: apps/v1
kind: Deployment
metadata: { name: vllm, namespace: workloads }
spec:
  replicas: 1
  selector: { matchLabels: { app: vllm } }
  template:
    metadata:
      labels: { app: vllm }
      annotations: { confidential.ai/cw: vllm }
    spec:
      enableServiceLinks: false
      securityContext:
        runAsNonRoot: true
        runAsUser: 1000
        runAsGroup: 1000
        seccompProfile: { type: RuntimeDefault }
      containers:
        - name: vllm
          image: ${VLLM_IMAGE}
          env:
            - { name: HOME, value: /cache }
            - { name: TMPDIR, value: /cache }
          ports: [{ containerPort: 8000 }]
          readinessProbe:
            httpGet: { path: "${VLLM_READY_PATH}", port: 8000 }
            periodSeconds: 5
            timeoutSeconds: 2
          securityContext:
            allowPrivilegeEscalation: false
            capabilities: { drop: ["ALL"] }
          volumeMounts:
            - { name: shm, mountPath: /dev/shm }
            - { name: cache, mountPath: /cache }
      volumes:
        - name: shm
          emptyDir: { medium: Memory, sizeLimit: 1Gi }
        - name: cache
          emptyDir: { medium: Memory, sizeLimit: 1Gi }
EOF_MANIFEST

kubectl create namespace workloads --dry-run=client -o yaml | kubectl apply -f -
kubectl apply -f vllm.yaml

confidential.ai/cw: vllm names the workload's Service intent: the operator creates the headless Service c8s-vllm.workloads.svc.cluster.local. Every covered pod receives mesh injection. The webhook adds three init entries: persistent c8s-mesh and c8s-cert sidecars, and the c8s-cert-wait gate. Only platform containers hold the private mesh credentials; vLLM serves plain HTTP on port 8000 inside the pod. The injector forwards the readiness probe through the mesh endpoint to the application's HTTP path. The rollout wait in Step 2 therefore includes the image's serving-readiness check.

The image may remain blocked until Step 2 admits it. Ordinary outbound TCP is captured by the mesh, so this deployment relies on its bundled model rather than a startup download.

2. Allow the vLLM image

Check CDS against its approved launch pin, current node policy, and operator keys before changing the allowlist:

c8s cds verify "$CDS_URL" --image-policy-file cds-pin.json \
  --served-policy-file nodes-policy.json --operator-keys operator.pub

Require exit code 0 and inspect warnings. In an HA deployment, signed Genesis can select a node set different from the measured configuration's default, and later signed commands can change it. Use the intended current set in nodes-policy.json.

c8s allowlist add "${VLLM_IMAGE##*@}" "$VLLM_IMAGE" \
  --url "$CDS_URL" --image-policy-file cds-pin.json \
  --operator-key operator.key

Record the name after added, then set it for the route:

export WORKLOAD_ENTRY='<PRINTED_ALLOWLIST_ENTRY>'
kubectl rollout status -n workloads deployment/vllm --timeout=10m
kubectl get pods -n workloads

Expect the vLLM pod to become Running with 3/3 ready: vLLM and the two persistent sidecars. A rollout timeout requires investigating pod events and the image's resource/runtime contract. It is not an attestation verdict.

The private signing key stays local; a signed token authorizes the write. This convenience entry admits the image with any command, arguments, environment, and mounts, and grants no egress exits. It does not prove the exact model or application configuration. For stricter entries, see Manage the allowlist.

For a future bootstrap, generate the component/workload seed before building cds-config.yaml. CDS seeds from that measured configuration; changing cluster Helm values does not rewrite the running external VM's seed.

3. Point the Load Balancer at vLLM

Check the router pods and the Service in their dedicated namespace:

kubectl get pods -n c8s-router
kubectl get svc -n c8s-router c8s-router

Require ready router pods and the prepared network path. This tutorial keeps the existing front-door configuration: the client reaches the router using its certificate's DNS name and trusts the retained mesh CA for outer HTTPS in Step 6. The same CA separately anchors the SDK's attested mesh identity. Keep TLS certificate validation enabled.

Read the existing route state before choosing whether to create the first manifest:

c8s routes show --url "$CDS_URL" --image-policy-file cds-pin.json \
  --operator-keys operator.pub

If this prints a verified manifest, omit --new from both commands below. The specific error no route manifest stored on CDS means the store is empty: use --new only when creating the cluster's first manifest. If a CDS reset lost earlier routes, restore the retained approved manifest before continuing. Stop for any other error.

Preview an exact route for /v1/chat/completions. It takes precedence over prefix routes for that endpoint and replaces any existing exact route at that host and path. Review that replacement and every retained route before publishing:

c8s routes add --url "$CDS_URL" --image-policy-file cds-pin.json \
  --operator-key operator.key --operator-keys operator.pub \
  --host "$CHAT_HOST" --path /v1/chat/completions --match exact \
  --address c8s-vllm.workloads.svc.cluster.local:8000 \
  --workload "$WORKLOAD_ENTRY" --new --dry-run

Review every route in the preview, then publish and read back the signed manifest:

c8s routes add --url "$CDS_URL" --image-policy-file cds-pin.json \
  --operator-key operator.key --operator-keys operator.pub \
  --host "$CHAT_HOST" --path /v1/chat/completions --match exact \
  --address c8s-vllm.workloads.svc.cluster.local:8000 \
  --workload "$WORKLOAD_ENTRY" --new

c8s routes show --url "$CDS_URL" --image-policy-file cds-pin.json \
  --operator-keys operator.pub

Confirm the stored host, exact /v1/chat/completions path, backend, and workload name match this deployment, then retain the printed digest:

export ROUTE_DIGEST='<ROUTE_DIGEST>'

Replace <ROUTE_DIGEST> with the reviewed manifest's 64 lowercase hex characters. This pin is required in Step 5. The router polls routes every 30 seconds by default.

The route's workload is the allowlist entry name, distinct from the vllm Service ID. The router authenticates the backend's CDS-issued mesh leaf and its workload stamp, requiring its allowlist digest to equal the stamped router's. The pod's mesh endpoint decrypts that connection and delivers ordinary HTTP to vLLM. After an allowlist change, requests can fail until the router and backend certificates carry matching policy digests.

4. Prepare the client pins

Keep the router identity, CDS identity, and mesh CA separate. Recheck the retained CA through the approved CDS pin without overwriting it:

c8s cds ca --url "$CDS_URL" --image-policy-file cds-pin.json \
  --out mesh-ca.candidate.pem
cmp mesh-ca.pem mesh-ca.candidate.pem

Require the CA read to succeed; cmp exit 0 confirms unchanged bytes. The CA command prints SHA-256 fingerprints of certificate DER on stderr. Investigate any difference through CDS verification before accepting a replacement. A singleton restart regenerates its CA and can lose nonpersistent routes; a surviving HA cluster can restore a restarting replica's state.

Create router-policy.json from the approved router-pin.json obtained through Azure policy preparation. For this AKS/AMD SEV-SNP path, copy the router-approved entries' measurements[].measurement values into the SDK's measurements array. The one-digest example is:

{
  "platform": "az-snp",
  "measurements": ["<APPROVED_ROUTER_NODE_LAUNCH_DIGEST>"],
  "requireCollateral": true
}

Replace the placeholder with the approved 96-hex SHA-384 value; include additional approved router-node digests as separate array entries. Use snp for native SEV-SNP evidence. These are router-node values, separate from the external CDS policy in cds-pin.json.

Copy the binary DER CRL supplied by the deployment operator:

cp '<OPERATOR_SUPPLIED_CRL_DER>' snp-crl.der

<OPERATOR_SUPPLIED_CRL_DER> is the file obtained through the prerequisite's validated vendor-collateral procedure. Have the operator confirm its processor generation against the router nodes' deployment inventory and provide a current CRL. Keep the DER bytes unchanged; Step 5 loads them as a Uint8Array and requires successful collateral verification.

For an existing native Intel TDX deployment, obtain its approved router-node manifest.json through launch measurement preparation. Copy mrtd, rtmr1, and rtmr2 from its tdx object (or its top level for a flat manifest) into this SDK policy. All three 96-character lowercase hex values must come from one build:

{
  "platform": "tdx",
  "tdxImage": {
    "mrtd": "<APPROVED_MRTD>",
    "rtmr1": "<APPROVED_RTMR1>",
    "rtmr2": "<APPROVED_RTMR2>"
  },
  "requireCollateral": true
}

Supply matching Intel PCS collateral as tdx-collateral.json in the SDK's tdxCollateral format, using strings for the verbatim signed JSON bodies and PEM certificate/CRL fields. The script below supports snp, az-snp, and native tdx; its collateral-required policy does not support az-tdx. SDK policy JSON is not a CLI --image-policy-file, and binary SNP collateral cannot be embedded in it as a JSON array.

These reference values come from approved build/deployment records, not from the endpoint being tested. A launch pin establishes only the state that platform measures; a bare AKS digest does not identify every guest or application file. Stock baked nodes leave TDX policy measurement off. For a supported deployment that enables it, prepare the additional history acceptance inputs before connecting.

5. Verify and connect

Create chat.mjs with this block, then append both JavaScript blocks from Step 6 to the same file. Keep the policy, collateral, and mesh-ca.pem beside it:

import { C8sClient } from "c8s-verify";
import { readFileSync } from "node:fs";

function requiredEnv(name) {
  const value = process.env[name];
  if (!value) throw new Error(`Set ${name} before running chat.mjs`);
  return value;
}

const policy = JSON.parse(readFileSync("router-policy.json", "utf8"));
if (policy.platform === "snp" || policy.platform === "az-snp") {
  policy.snpCrl = new Uint8Array(readFileSync("snp-crl.der"));
} else if (policy.platform === "tdx") {
  policy.tdxCollateral = JSON.parse(readFileSync("tdx-collateral.json", "utf8"));
} else {
  throw new Error("This tutorial requires snp, az-snp, or native tdx evidence");
}

const client = new C8sClient({
  ...policy,
  baseUrl: `https://${requiredEnv("CHAT_HOST")}`,
  meshCaPem: readFileSync("mesh-ca.pem", "utf8"),
  routeDigests: [requiredEnv("ROUTE_DIGEST")],
  requireCollateral: true,
  allowPartial: false,
});

const session = await client.connect();
console.error(`Verified router: ${session.attestation.measurement}`);
for (const warning of session.attestation.warnings) console.error(warning);

connect() generates a nonce and an X-Wing keypair (X25519 + ML-KEM-768), posts the nonce and encapsulation key to /.well-known/c8s/attest-pq, and verifies the returned evidence and transcript. It checks the router policy, mesh CA, route digest, and supplied collateral before decapsulating and deriving the client channel. Failed checks return no session; do not bypass them or enable partial results to continue. See what it verifies.

6. Chat over the sealed channel

Append the request function. The model ID matches the preconfigured image from Step 1:

async function ask(messages) {
  const res = await session.fetch("/v1/chat/completions", {
    method: "POST",
    headers: { "content-type": "application/json" },
    body: JSON.stringify({ model: "tutorial-model", messages, stream: false }),
  });
  if (res.status !== 200) throw new Error(`vLLM HTTP ${res.status}: ${res.text()}`);
  return JSON.parse(res.text()).choices[0].message.content;
}

Append the chat loop:

import { createInterface } from "node:readline/promises";
import { stdin as input, stdout as output } from "node:process";

const rl = createInterface({ input, output });
const history = [];
try {
  for (;;) {
    const prompt = await rl.question("you> ");
    if (!prompt || prompt === "/exit") break;
    const messages = [...history, { role: "user", content: prompt }];
    const reply = await ask(messages);
    history.push({ role: "user", content: prompt }, { role: "assistant", content: reply });
    console.log(`bot> ${reply}\n`);
  }
} finally {
  rl.close();
}

Run from the directory containing the files, with CHAT_HOST and ROUTE_DIGEST from earlier steps still set:

NODE_EXTRA_CA_CERTS=mesh-ca.pem node chat.mjs

Expect Verified router: …, then you>. Send a short prompt; a bot> reply means an application request and response traversed this verified session. Enter /exit to finish. If connection or inference fails, resolve the reported failure before sending another prompt.

session.fetch seals the method, path, headers, and body; the router decrypts them, routes to vLLM, and seals the response. The tunnel buffers a complete response, so this example sets stream: false. Ordinary HTTPS routes can stream, but their requests do not use this sealed session. A successful standalone CLI verification does not bind a later ordinary HTTP request.

Route or router policy changes can reject later requests and end the session. The SDK reports outer non-2xx failures as channel_error; it neither reconnects nor replays requests. Review any changed pins, then restart the client to establish a fresh session. Checks on a later request do not cancel an already running exchange.