Limitations

Current enforcement boundaries, volatile-state availability limits, and platform constraints in C8s.

C8s protects workloads within a measured TEE, with enforcement that depends on the node image, CDS configuration, and client verification policy. These are the boundaries to account for when deploying and verifying it.

Trust and enforcement gaps

  • Launch pins identify the trusted policy. CDS verifies its required configuration against its own launch evidence. That configuration binds operator keys, initial workloads, node policy, node address ranges, and certificate naming policy. An empty node policy refuses issuance. For singleton CDS, the configured node pins define the node policy. They also supply the default node policy for HA Genesis, the signed cluster initialization. Genesis can select a different initial set, and later signed commands can change it. Clients need the intended complete CDS image and configuration pins. See the trust root.
  • Mesh authentication has a narrower scope than hardware verification. The injected mesh checks TLS 1.3, the CDS-issued chain, certificate purpose and validity, the mesh protocol, and a sandbox ID. It does not reverify hardware evidence on each connection. The sandbox ID is vouched for by the CDS using the node's admission inventory; it is not part of the hardware report. The measured node and its trusted platform components remain inside this boundary.
  • The mesh has no per-workload peer access lists. Its production endpoints accept a valid peer identity from the same mesh CA without a destination-specific workload authorization callback. Named routing is a separate check. Workload identity uses the CDS certificate's sandbox and workload information, not a SPIFFE URI scheme. Private mesh credentials belong to injected platform containers; application mounts of that credential volume are refused.
  • Container policy is not a signature over the complete pod specification. The allowlist constrains image digests, argv, environment, and mounts according to each entry. Open fields remain open. Secret release and signed egress filter injected mesh and credential containers from the observed pod set. The remaining workload containers must satisfy every declared main-container policy, and each must match a declared main or init policy; a container that matches none is foreign. Matching uses the container's policy fields, not its init-versus-main lifecycle identity, so an observed init container can satisfy a declared main policy. This is not a general pod-spec integrity check. See the image allowlist.
  • Runtime hardening depends on the deployment. Baked nodes enforce a measured runtime floor for covered containers, including nonroot execution, bounded capabilities, no_new_privs, and private namespaces. Their runtime refuses checkpoint restore and attachable launches; exec access depends on the image's locked or debug mode. Trusted platform launches have scoped exceptions. Managed deployments use chart-managed NRI policy and do not inherit these baked-runtime guarantees.
  • Signed exits authorize destinations, not payloads. Egress grants require pinned argv and mount rules; env: any is still permitted. The observed containers must uniquely satisfy the declaring workload's main policies before exits open, and foreign additions are refused. An exit bypasses mesh capture without encrypting or inspecting application data. Changes or removal of exits require pod recreation.
  • Policy-history measurement is optional. It uses TDX runtime registers and is a no-op on SNP. The stock baked image leaves it disabled, and the chart cannot enable it on baked nodes. Stock baked nodes therefore do not provide an attested history of every applied allowlist. Where measurement is supported and enabled with the required mounts, a measurement failure or journal-capacity refusal leaves the previous policy in force.
  • Operator authority is tied to the CDS launch configuration. Replacing operator keys requires a new measured configuration and corresponding client pins. HA mesh-CA key rotation does not rotate operator authority. See Authorizing mutations.
  • Revocation has several independent boundaries. A policy update can stop denied running containers only when enforcement is outside audit mode and enforce_existing is enabled; exemptions still apply, and stop failures are logged. Expired or withdrawn mesh credentials make the endpoint unready and refuse new forwarding. These checks do not guarantee immediate termination of established streams. A mesh certificate-chain check also does not establish hardware endorsement-key revocation status; that depends on the attestation verifier and its collateral policy.
  • Mutable storage lacks authenticated integrity. Mutable encrypted volumes use plain dm-crypt without dm-verity, so host bit flips and rollback are not detected by an integrity layer. Immutable volumes verify blocks against their verity root. See What this defends.
  • Namespace exemptions differ by control. Webhook injection excludes the release and router namespaces, kube-system, kube-public, kube-node-lease, configured webhook.extraExcluded namespaces, and local-path-storage on non-baked nodes. The host-namespace admission policy excludes the release and router namespaces, kube-system, configured hostNamespacePolicy.exemptNamespaces, and local-path-storage on non-baked nodes. Those admission exemptions do not disable the baked runtime floor. Setting hostNamespacePolicy.enabled=false removes that Kubernetes admission policy; guest runtime enforcement remains a separate control.

Availability and operations

  • CDS supports singleton and replicated volatile state. A singleton generates a new mesh CA at startup and loses its in-memory secrets and sandbox bindings on restart. Its volatile allowlist store resets to the configured workloads. With measured HA configuration and an explicit signed Genesis, replicas share the CA private key, secrets, allowlist, and ledger state. A replica restart is not inherently loss of cluster state: it can rejoin a serving cluster. See Mesh CA bootstrap.
  • Replication requires quorum and retains no durable copy. Before serving, a replica needs a leader-confirmed read lease and must apply the log through the position that lease identifies. Quorum loss stops serving after lease validity ends; it does not erase surviving memory. Loss of every replica's volatile state loses the cluster's secrets and keys. Volume keys then require an operator's escrow copy. See Application secrets.
  • Survivor recovery is not operationally implemented. The Recover CLI can submit a signed decree that a replica reports as held: retained in memory but not applied. The runtime acts on held Genesis and Admit decrees, but does not act on Recover. It also has no automatic mechanism to stop serving in response to a suspected competing recovery. Permanent quorum loss therefore has no working survivor-Recover procedure in this release. Preserve surviving VMs on a halt; restarting them discards their memory.
  • Replicated state and history are bounded. The 256 MiB decoded-state limit covers more than secrets and reserves admission headroom; it is not usable secret capacity. Writes can fail for unavailable quorum or state capacity. HA policy-history retention is bounded, and pruned objects and signatures are unavailable as an audit archive.
  • CA-key replacement requires pod recreation. Certificate renewal under the same CA key preserves the pod's issuer binding. An actual mesh-CA key change leaves existing pods bound to their original issuer; new cross-key connections fail. Established streams are not guaranteed to stop immediately.
  • Secret-dependent initialization can block secret release. Secret release needs observed containers satisfying all declared main-container policies. An ordinary init container that waits for the secret can block initialization when the observed set does not yet satisfy those policies. Matching uses policy fields, so an observed init container can itself satisfy a declared main policy. See Application secrets.

Platform constraints

  • GPU checks are scoped to baked-node boot. With NVIDIA GPUs present, the baked node checks CC mode and verifies GPU evidence before enabling GPU CC Ready and starting the node. This is a boot check, not continuous GPU-state verification. CDS certificate issuance does not require GPU evidence, so a CDS-issued mesh leaf alone does not prove GPU protection to a relying party. Trusted node-defined NVIDIA CDI edits remain part of the platform's admission boundary.
  • External service meshes are unsupported and unvalidated. C8s injects its own pod endpoint and uses pod-local nftables capture. Istio, Linkerd, and other interception or mTLS layers need integration validation; C8s does not establish their combined behavior or trust boundary.
  • Capture has explicit exceptions and check points. NRI verifies mesh namespaces and rules at admission and identity assertion, not continuously. DNS, local/reply/control traffic, health checks, platform roles, and signed exits have scoped plaintext exceptions. Unsupported traffic is dropped. Root-owned sockets entering a pod namespace are blocked, including the ordinary port-forward path; this is not a claim about every forwarding API.

Browser and out-of-cluster verification

  • Verification requires caller-selected policy and collateral. The c8s-verify client accepts a minimum SNP TCB, an SNP CRL, TDX collateral, and requireCollateral. The caller supplies these inputs; a successful image-pin or mesh-identity check alone does not establish firmware freshness or revocation.
  • The over-encrypted channel buffers complete messages. Its 8 MiB request-record cap covers the application body, method, path, and headers plus encoding and encryption overhead. Its buffered backend-response cap is 32 MiB. Request and response buffers share a reservation subject to aggregate and per-client memory limits. Direct application requests have a separate body limit, defaulting to 1,048,576 bytes: the chart uses router.routing.maxRequestBodyBytes, while baked nodes use the signed launch field router.maxRequestBodyBytes. The tunnel endpoint uses its 8 MiB record limit instead. Oversized request bodies receive 413; oversized backend responses produce a sealed 502 error. Raising the application-body cap does not change the tunnel cap or enable tunnel streaming. Large transfers need application-level range or chunk APIs.
  • Attested sessions do not follow routing changes transparently. Route or router digest changes refuse later requests and require a new attested connection or session. These admission checks do not stop already running streams. The JavaScript client reports an outer non-2xx response as channel_error and does not reconnect or replay automatically.

Out of scope for this milestone

  • Integrity of the complete pod specification beyond the enforced container, runtime, and mesh rules.
  • Multi-tenant isolation between workloads on the same node, and federated multi-cluster control planes.