Integrating into routers

Integrate Confidential Inference as an OpenAI-compatible upstream with two gates: review and approve releases ahead of deployment, then verify the transport that carries each prompt. Your router holds the approved measurements; hardware attestation binds the checked software identity to the transport keys.

Place verification before prompt forwarding

Customer
   │
   ▼
Your router + upstream verifier
   │  Verify the connection before sending the API key or prompt
   ▼
Confidential gateway → inference router → inference workers

This integration assumes your router terminates the customer's connection and opens a separate connection to Confidential Inference. The upstream verifier protects that second connection. If your router also runs in a TEE, its own attestation can provide customers with assurance about the router as well. Routers that pass the customer's TLS connection through unchanged place upstream verification at the customer client instead.

Inside the service, c8s provides the measured execution environment, workload admission, and authenticated mesh. Read Components & Data Flow for the internal path and trust boundaries for what a confidential VM isolates. The Inference API reference documents authentication, models, completions, and streaming.

Review a release before it reaches production

First arrange a release-evidence handoff with Confidential AI. Request the candidate's signed release bundle, source revision, image artifacts, build manifests, exact canonical allowlist bytes, deployment operator public key, and route policy with its expected digest or signing public keys. Also request the deployed release's supported attested transport and streaming contract, including the verifier and configuration needed to use it.

Obtain these inputs ahead of deployment and approve them through your security review process. A release identifier names the candidate; it is not itself proof of its contents.

Review every component that can receive prompts or influence where they go: the gateway, inference router, workers, and the c8s components that enforce policy and issue identities. Include logging, telemetry, model loading, and outbound connections in that review. Attestation identifies software; deciding whether that software preserves confidentiality is your review decision.

Use the following evidence to connect that decision to the running service:

Review and pin the applicable policy and operator authority in addition to image measurements. Use the production v0.16.2 verification profile, including the deployment's RTMR3 pin. The generic verification model explains the policy checks.

For Intel TDX, approve MRTD, RTMR[1], and RTMR[2] together from the same build. MRTD alone identifies the firmware without identifying the full guest image. See whole-image matching. If the release uses additional measured state to establish its policy, require that binding too; a base-image match cannot substitute for a policy check.

Record approved measurement tuples in your router's verification policy. The measurements-config guide explains how to represent complete images and overlapping releases. Its cluster-install commands are for c8s operators; configure your upstream verifier with the equivalent approved set. Deriving values from a manifest reads the publisher's claims—it does not independently rebuild or re-measure the artifacts.

Pin these reviewed measurements, policy, and operator authority in the router's verifier. End-user clients that verify the service directly need the same trust inputs and request-transport checks.

Authenticate the CDS and approve its policy authority

The CDS (Certificate Distribution Service) issues the identities used by the c8s mesh. Review the CDS software, image measurements, and operator authority alongside the workload policy. Its certificate authenticates issued identities; your policy pins define which software and authority you accept.

Accept a presented CDS public key only after verifying hardware evidence that both matches an approved launch identity and cryptographically binds that key. Then verify the certificate chains and identity proofs that connect it to the service endpoint. Merely receiving a certificate alongside a valid report is insufficient: the key must be included in the report's verified binding.

See verifying the CDS itself for the hardware-to-key binding and the attestation flow for certificate issuance.

A certificate change requires fresh verification. Accept it only when your approved measurements, policy, key bindings, and identity checks all pass. If you pin a specific mesh CA, update that pin through your approval process. A familiar certificate does not excuse a failed measurement or evidence check.

The generic JavaScript verifier requires either a mesh CA pin for a specific cluster, canonical allowlist bytes for a deployment class, or both. A deployment class accepts instances matching the approved image and workload policy without identifying one particular cluster. Request the exact canonical document bytes in the release-evidence handoff; preserve them without parsing and re-serializing the JSON, because the verifier hashes those bytes. Image measurements remain required in either case.

A transcript-committed CA is the issuing mesh CA whose certificate digest is bound into the attested session transcript. It authenticates the presented mesh chain, but that binding does not independently attest the CDS software that issued it. Keep the CDS verification and workload-policy checks in your approval process. Neither image measurements nor deployment-class acceptance establishes service ownership.

Gate each upstream connection

For production v0.16.2, follow the inference attestation guide: use the reviewed signed release, exact allowlist, deployment operator key, full node measurements and RTMR3 pin, TLS name, and expected front-door workload. Require Verified, Intel TCB status UpToDate, and an operator-signed route. Send requests through TEErminator's local endpoint as that profile directs.

Confirm the deployed release and transport contract with Confidential AI. The protocol comparison below explains generic c8s bindings; it does not establish which protocol TEErminator uses or replace the production profile's required pins.

The c8s transports have different bindings:

These are c8s protocol endpoints, not a claim that the hosted inference gateway exposes them. Its public evidence interface is documented separately below. Stop the integration if the provider cannot supply a supported transport that meets the binding requirements below, including incremental streaming if you require it. Enable customer traffic only after that transport passes validation.

Configure the verifier to require all of the following before forwarding an API key or prompt:

1. Valid TEE evidence under trusted hardware-vendor roots, with the verifier's required security-version, validity, and debug-policy checks.
2. A complete match against one manually approved release's measurements.
3. Evidence that the reviewed workload policy governs the protected path, including any operator authority that can change it.
4. The CDS public-key binding and the relevant certificate and workload-identity proofs, including the expected gateway identity and TLS name.
5. Freshness for the live evidence exchange and a binding to the transport that will carry the request: the inner channel for attest-pq, or the observed TLS leaf on the retained connection for attest-lb.
6. The operator-signed route required by the production profile. A generic c8s integration may instead pin an approved route digest where its verifier supports that policy; the JavaScript options in the review table describe that generic configuration.

The current hosted API contract provides public /allowlist and /v1/discovery endpoints without an API key. /allowlist serves the active workload policy; /v1/discovery provides connection metadata, certificate information, and TDX evidence. Verify the connection and compare the exact allowlist bytes with the reviewed signed release policy, as the production profile directs. Downloaded JSON alone does not complete verification or protect a later ordinary HTTPS request. Front-door verification does not provide a separate receipt from every inference worker or prove current worker health or execution of a particular request.

Keep inference requests on the verified transport, including pooled connections, reconnects, and retries. Re-establish and verify expired inner sessions; verify every new TLS connection when using attest-lb. After route_digest_changed, router_digest_changed, or binding_expired, attest again even if the underlying connection remains open. Check the newly presented route and workload policy against your approved set before retrying; a previously approved launch measurement does not approve a changed policy. These refusals apply to subsequent requests. The production profile uses certificate and verdict-cache lifetimes; a policy update does not guarantee immediate disconnection of an open stream.

The generic c8s inner tunnel buffers responses. For incremental chat-completions streaming, validate a service-supported transport that both streams responses and preserves its attestation binding.

The operator verification guide is useful for investigation and release checks. The CLI's attest-pq check discards its keypair without opening an application channel. Its attest-lb check verifies the observed serving leaf but does not hand an application the checked connection. Neither verdict authorizes a later ordinary fetch. Application traffic must use the verified inner channel or a TLS connection verified in its request path.

Approve upgrades without trusting new values on first use

Agree on a release-review process with Confidential AI, including a review window for each new measurement set before deployment. Use that window to review the candidate artifacts and prepare your verifier's approved set before traffic reaches the new release. During a rolling upgrade, accept the old and new releases as separate, complete measurement entries; never mix registers across releases.

Removing a measurement from policy must also retire connections authenticated under it. Otherwise a long-lived connection can outlast the approval that created it. Keep release approvals separate from the upstream's live responses: never automatically add a newly observed measurement to restore availability.

Validate the integration before enabling customer traffic

Exercise the verifier with non-sensitive requests and observe both acceptance and rejection. Require these outcomes:

• An approved release completes a request over its verified transport; if you require incremental streaming, verify that separately on the selected transport.
• An unapproved measurement or a mismatched CDS key prevents prompt forwarding.
• A wrong gateway identity, stale nonce, or invalid connection binding prevents prompt forwarding, even when other evidence is valid.
• A certificate change passes only with valid evidence, an approved release, and any required mesh CA pin update.
• Reconnects and retries repeat verification, and retirement closes old connections.
• On generic c8s bound channels, a change to the bound route digest or the router certificate's stamped allowlist digest, or expiry of the binding, rejects subsequent requests until fresh attestation passes. A newly presented policy outside the approved set prevents retries from forwarding prompts.

Validate production policy-update behavior against the documented certificate and verdict-cache lifetimes. A change to the active CDS policy does not by itself change the allowlist digest stamped in an existing router certificate.

Keep fallback providers subject to the same verification requirements the customer selected.

Enable customer traffic only after these checks pass for the deployed release. For the scope of that assurance, see inference attestation and the applicable c8s limitations. GPU evidence, model-weight identity, and output quality are separate from the launch and admission checks described here.