Serve a model with KServe

1. Check the supplied predictor shape

The operator's validation record must show that the selected controller reconciles iris.yaml and that the packaged model loads successfully. Capture its generated Deployment:

export VALIDATION_CONTEXT='<OPERATOR_VALIDATED_CONTEXT>'
kubectl --context "$VALIDATION_CONTEXT" get deployment iris-predictor -n workloads \
  -o json > iris-predictor.json

The Deployment's spec.template.metadata.annotations["confidential.ai/cw"] must equal iris, and its generated Pods must inherit that annotation. Validate propagation through the selected KServe API in the supplied record. C8s's workload controller watches the Deployment template and creates headless Service c8s-iris; annotating a standalone Pod does not create that Service. The annotation names Service intent. All covered pods receive the mesh independently of it.

Compare policy-relevant fields with the approved target artifacts: application init/main container membership, platform-specific immutable image digests, resolved commands/arguments, effective OCI environments, and mount rules/provenance. Generated object names, resource versions, and unrelated labels are not the identity policy. Pod JSON is also accepted by derive, but it does not replace the Deployment-template check above.

Review every generated application container, including any local initializer, for:

• Non-root execution, disabled privilege escalation, allowed capabilities, private process namespaces, and the deployment's admission controls. On baked nodes the measured runtime also enforces its security floor.
• Supported HTTP health probes with harmless paths, resolvable application ports, no custom headers, and timeouts at most five seconds. TCP/gRPC probes and HTTP lifecycle hooks are refused; exec probes depend on the node's exec mode.
• Approved storage and local access to the packaged model. Record its content digest and fixture results. This flow does not fetch from an external bucket or grant external model download egress. A public model tests the application path, not secret model delivery.

2. Admit the complete predictor

Use the operator-supplied iris-env.json and iris-mounts.json, each mapping every application init and main container name to its approved policy. Use exact or match environment policies as appropriate, and exact mount rules for reviewed volumes. Derive one named entry:

export CDS_URL='https://<CDS_HOST>:<CDS_PORT>'

c8s allowlist derive iris-predictor iris-predictor.json \
  --env-file iris-env.json \
  --mounts-file iris-mounts.json > iris-entry.json

c8s allowlist lint iris-entry.json --strict
c8s allowlist apply iris-entry.json \
  --url "$CDS_URL" \
  --image-policy-file cds-pin.json \
  --operator-key operator.key

derive emits a bare map with the key iris-predictor, pins argv using the image configuration when needed, and drops C8s-injected containers while naming them on stderr. It does not infer mount provenance, full environment policy, or egress. With no findings, lint prints ok: no findings; a successful apply prints applied iris-predictor. HTTPS commands use the external CDS VM address and its launch-file-bound pin.

CDS matches the complete sandbox admission history to one unambiguous entry, including initializer records retained after exit. Every recorded application container must fit it, and every required main-container policy must appear in that record. This establishes launch identity, not current liveness. Keep conflicting broad entries out of the effective policy.

3. Deploy a confidential InferenceService

Apply the supplied manifest to the target cluster only after the prerequisite checks pass:

kubectl apply -f iris.yaml

Wait for the validated controller to create iris-predictor, then inspect it:

kubectl get deployment -n workloads iris-predictor -o json > iris-target.json
kubectl get inferenceservice -n workloads iris
kubectl get pods -n workloads -l serving.kserve.io/inferenceservice=iris

Compare iris-target.json with the approved fields from Step 1 before continuing. If the controller changes images, argv, environment, mounts, or container membership, stop and have the operator reconcile the artifacts and policy. Inspect admission events if creation fails; a missing Deployment must be resolved through the validated controller topology.

Inspect the generated pod and Service:

kubectl get pod -n workloads -l serving.kserve.io/inferenceservice=iris \
  -o jsonpath='{range .items[0].spec.initContainers[*]}{.name}{"\n"}{end}'
kubectl get svc -n workloads c8s-iris

The injected init-container order begins c8s-mesh, c8s-cert, c8s-cert-wait, followed by any approved application init containers. The Service has clusterIP: None. Application containers cannot mount the platform's private credential volume. The mesh captures application TCP to Service destinations as well as pod IPs; use the headless Service here as an explicit predictor backend.

Proceed only when the supplied packaged-model startup checks and actual readiness checks pass. Kubernetes readiness alone does not prove the predictor's CDS workload stamp.

4. Point the front door at the predictor

Use the router access provisioned with the cluster:

kubectl get svc -n c8s-router c8s-router
export ROUTER_ORIGIN='https://<REACHABLE_ROUTER_HOST>'
export ROUTE_HOST='c8s-router.c8s-router.svc'

For a release named c8s, the default router SAN is c8s-router.c8s-router.svc. Adjust the Service name and routed host for your release or explicit router.san configuration. The external origin and sealed request host can differ. Use the deployment's reachable ingress; port-forwarding into a covered pod is blocked by its mesh rules.

After validating that the predictor listens on HTTP port 8080, store the route:

c8s routes add \
  --url "$CDS_URL" \
  --image-policy-file cds-pin.json \
  --operator-key operator.key \
  --operator-keys operator.pub \
  --host "$ROUTE_HOST" \
  --path / \
  --address c8s-iris.workloads.svc.cluster.local:8080 \
  --workload iris-predictor \
  --new

c8s routes show \
  --url "$CDS_URL" \
  --image-policy-file cds-pin.json \
  --operator-keys operator.pub --raw > iris-routes.json
export ROUTE_DIGEST=$(c8s routes digest iris-routes.json)

Use --new for the cluster's first manifest; omit it when updating an existing one. The command uses prefix matching by default and replaces a route with the same host, path, and match kind. Preserve and review other routes when updating a shared manifest. routes show --raw writes the verified manifest bytes; routes digest prints their SHA-256 hex digest for the client to pin. Review that saved manifest before distributing the pin.

--workload iris-predictor names the complete allowlist entry. It is independent of the iris Service intent. Router authenticates the backend's mesh certificate, requires its named workload stamp, and, for a stamped router, checks equality with its own allowlist digest. The injected endpoint terminates mesh TLS and forwards plaintext to the predictor's HTTP socket. An allowlist update changes the digest, so an already-issued router or backend leaf can refer to the preceding policy until renewal. The credential agent watches CDS policy changes and renews automatically; with the default watch enabled it checks every minute and schedules policy renewal with jitter. An initially unnamed predictor also retries for a named stamp.

Inspect renewal and routing failures with:

kubectl logs -n workloads deployment/iris-predictor -c c8s-cert --since=10m
kubectl logs -n c8s-router deployment/c8s-router -c c8s-cert --since=10m
kubectl logs -n c8s-router deployment/c8s-router -c cds-attest --since=10m

The routing error backend allowlist digest differs from the router's identifies a stamp mismatch. Allow the configured renewal loops to converge, then open a new client session and retry only the non-sensitive fixture in Step 5. If renewal fails or the mismatch persists, stop and give the operator these logs to resolve CDS access, policy matching, or disabled watch settings. Do not mount or manually replace platform credentials in the application.

5. Verify the enclave and predict

Check the router against its trusted image policy and cluster authority:

c8s verify "$ROUTER_ORIGIN" --kind lb --mode attest-pq \
  --image-policy-file router-pin.json \
  --mesh-ca mesh-ca.pem \
  --route-digest "$ROUTE_DIGEST"

attest-pq also works with a WebPKI outer TLS endpoint. Review the verdict, warnings, and policy checks in Verify a deployment. This router check does not execute a prediction or validate the model-storage path.

The following client example is for an Azure SEV-SNP router. It reads the operator-supplied router-az-snp-measurement.txt from the prerequisite bundle; for TDX use the image tuple and platform settings in the client reference. Keep the mesh CA and the reviewed route digest as separate pins.

// predict.mjs
import { C8sClient } from "c8s-verify";
import { readFileSync } from "node:fs";

const client = new C8sClient({
  baseUrl: process.env.ROUTER_ORIGIN,
  platform: "az-snp",
  measurements: [readFileSync("router-az-snp-measurement.txt", "utf8").trim()],
  meshCaPem: readFileSync("mesh-ca.pem", "utf8"),
  routeDigests: [process.env.ROUTE_DIGEST],
});

const session = await client.connect();
const res = await session.fetch("/v1/models/iris:predict", {
  method: "POST",
  headers: {
    "content-type": "application/json",
    host: process.env.ROUTE_HOST,
  },
  body: JSON.stringify({
    instances: [[6.8, 2.8, 4.8, 1.4], [6.0, 3.4, 4.5, 1.6]],
  }),
});
if (res.status < 200 || res.status >= 300) {
  throw new Error(`Predictor returned HTTP ${res.status}: ${res.text()}`);
}
console.log(res.text());

node predict.mjs

The operator-supplied manifest and validation record must use this request path and schema. The program prints the actual response body; its class values depend on the approved model artifact. Compare them with a known test vector from that artifact before using private input. connect() must pass its attestation and pin checks before session.fetch() sends the sealed prediction request.