c8s Is Now Generally Available

Back in April, we released our whitepaper detailing a Confidential Kubernetes based on Trusted Execution Environments (TEEs). Today we're launching the first general availability release of our implementation into the wild.

c8s allows you to run workloads such as inference, training, or processing of customer or proprietary data privately, such that your dev team, your system administrators, and your cloud provider cannot see any of the runtime data. You can prove this cryptographically.

To do this, we leverage TEEs: a secure space in hardware where code executes encrypted from the host machine that it's being run on. Despite having been around for more than a decade, nobody has built convenient tooling that spans the entire stack from bare metal to end user for use and verification — until now!

How do TEEs work?

TEEs are a hardware-embedded mode for your CPU for launching confidential virtual machines (CVMs). These CVMs create a secret key on launch that they use to encrypt the memory they use, such that the host OS can't read anything in it. They also add checksums to each memory region so if the host OS tries to tamper with it, the CVM detects it and can panic.

CVMs can also request a signature from the manufacturer's key burned onto their chip to attest that they launched a specific VM image inside legitimate secure hardware. External parties or software can verify these attestations before sending any sensitive data into the CVM, gaining high confidence that the data is secure.

The CVM becomes a trust boundary — anything running inside it can be trusted. Anything running outside it cannot be trusted, and anything crossing the boundary must be encrypted with a key that lives inside the trust boundary.

How does Kubernetes work?

Kubernetes is a container orchestration tool that automatically provisions and scales compute resources.

If we are running an online store in a cloud provider, our traffic can change drastically based on things like the time of day or whether we're running a sale. If we over-provision resources to handle the peak of our traffic, they will stand idle when traffic is low. Kubernetes solves this by spinning up and down machines on cloud providers based on traffic, usage, and other metrics.

It's become the de facto standard for managing cloud infrastructure at scale and is popular for bare-metal deployments too.

Typically, your cloud provider runs the control plane for Kubernetes on your behalf. This control plane decides which containers to run where. When the control plane wants to start a new container on a node, it adds the container to its state served by its API server. The kubelet on the node polls the API server and identifies a drift between the node's state and the state desired by the control plane, and starts/stops the necessary containers to reach parity. This puts a lot of power into the hands of your control plane!

How does c8s combine these for confidentiality?

In c8s, we run individual pods or entire nodes inside CVMs to create a trust boundary. Though we leave the control plane outside this trust boundary, we run a shim for the container runtime that refuses to start any container that isn't on an allowlist managed by the operator. This means that untrusted workloads can't even start inside a c8s cluster!

After starting, workloads perform mutual attestation between one another to establish TLS sessions before any communication happens, so that all intra-cluster communication is encrypted.

Operators verify the attestations of the core components and the state of the cluster after installation.

Users interacting with the cluster form a root of trust by verifying the attestation of the Certificate Distribution Service and validating the allowlist that it's enforcing inside the cluster. From that, they can create a chain of trust with other components and be sure that their data is secure and confidential during processing.

What's the performance like?

On CPU, the performance penalty for confidential computing is already less than 5% for most workloads: the CPU enters a different mode, but much of the normal operation continues.

Until recently, GPU performance paid a 20% penalty for confidential operations. After a bunch of research and optimization work, our team has achieved a reduction of the overhead from 20% to 1%!

We'll be releasing a paper on how we did it in the next few weeks.

How can I use it today?

The code is available in the c8s repository on GitHub. We've provided a variety of tutorials in our docs, such as:

• Spinning up a confidential VM on a cloud provider and installing c8s on it
• Running vLLM inside c8s and attesting its confidentiality end-to-end
• Deploying NVIDIA Dynamo inside a c8s cluster for serving distributed inference with confidentiality

Licensing

For launch, we've licensed the code under AGPL. This is a copyleft license and means that all installations and derivations of the work must also be licensed under the AGPL.

This is not our final license.

We wanted to get c8s into the hands of users as soon as possible, and we haven't figured out our long-term licensing strategy yet. We are sure that we want to make the license more permissive, but there are a host of different options and we don't want to make the wrong decision.

If you have a use case that's blocked by AGPL in the short term, talk to us and we can figure out a temporary license that can work for you.

Roadmap

A lot of work has gone into the c8s launch, but there's still a ton of work in flight to make it even better. In a few weeks, we'll be launching a key management system that's fully integrated into c8s so you can gate secret release by attestation: only specific workloads that can prove their code and prove they're running inside secure hardware can get an API key, model weight decryption key, or other secret.

It will behave like and proxy to the secrets managers you're used to, but with the cryptographic benefits of trusted execution environments.