Call Confidential Inference from LiteLLM

Route LiteLLM to Confidential Inference with the openai/ provider prefix and api_base https://api.confidential.ai/v1, from the Python SDK or the LiteLLM proxy config.

LiteLLM reaches Confidential Inference through its generic OpenAI-compatible provider: prefix the model ID with openai/ and set api_base.

Prerequisites

Install LiteLLM

pip install litellm

Call it from Python

import os
from litellm import completion

resp = completion(
    model="openai/deepseek-ai/DeepSeek-V4-Flash-0731",
    api_base="https://api.confidential.ai/v1",
    api_key=os.environ["CONFIDENTIAL_API_KEY"],
    messages=[{"role": "user", "content": "Explain remote attestation in one sentence."}],
)
print(resp.choices[0].message.content)

Everything after openai/ is the exact model ID from GET /v1/models.

Or add it to the proxy

In the LiteLLM proxy config.yaml:

model_list:
  - model_name: confidential-deepseek-v4-flash
    litellm_params:
      model: openai/deepseek-ai/DeepSeek-V4-Flash-0731
      api_base: https://api.confidential.ai/v1
      api_key: os.environ/CONFIDENTIAL_API_KEY

Clients of the proxy then request confidential-deepseek-v4-flash.

Verify before sending sensitive input

LiteLLM doesn't check attestation. Fetch /attestation with a fresh nonce and verify it first: Attestation.

How you know it worked

The call returns a chat completion. A 401 means the key is missing or wrong.

The LiteLLM proxy reads prompts in plaintext before forwarding them over TLS. Run it on infrastructure you trust, or inside a confidential VM, so the proxy doesn't widen your trust boundary.