IM Engineer Study Notes
← Back to list
Security & Privacy
#ConfidentialContainers#CoCo#ConfidentialComputing#TEE#Kubernetes#KataContainers#RemoteAttestation#KeyBrokerService#PeerPods
Last updated · 2026-09-27

Kubernetes Workload Protection with Confidential Containers (CoCo)

1. Overview

A. Definition

Confidential Containers (CoCo) is a cloud-native confidential-computing approach that runs a Kubernetes Pod inside a hardware-backed Trusted Execution Environment (TEE), protecting data in use and workload code from the host infrastructure, hypervisor, and other tenants.

Ordinary containers isolate processes and filesystems, but they normally share a host kernel. A cloud operator or host administrator with strong privileges may be able to inspect memory, copy image layers, or use debugging interfaces. Disk and transport encryption do not protect plaintext while an application is processing it in memory.

CoCo combines the lightweight virtual-machine isolation of Kata Containers with hardware memory encryption and integrity features such as AMD SEV-SNP and Intel TDX. A remote verifier checks measurements before an image-decryption key or secret is released. The result is not merely a container option but a workload trust chain connecting isolation, measurement, verification, and secret delivery.

B. Background and need

Cloud computing often separates the infrastructure provider from the data owner. A bank may analyze data in a public cloud, or several companies may execute a joint model without giving the operator unrestricted access to their inputs. CoCo offers a way to keep Kubernetes operations while moving sensitive data and model weights outside the infrastructure trust boundary.

Generative AI and data collaboration require protection for prompts, model weights, and intermediate results as well as input data. Signed images establish provenance and integrity, but do not hide image contents. Encrypted images, remote attestation, and conditional key release therefore need to be designed as one lifecycle.

CoCo also aims to preserve standard Kubernetes workflows. A workload can select a protected runtime with runtimeClassName, use OCI images, and remain subject to ordinary scheduling and admission controls. This does not make every Pod safe automatically: the control plane, registry, KBS, logs, and storage require separate trust-boundary decisions.

C. Scope

The first objective is confidentiality and integrity for data in use. Storage encryption protects data at rest, TLS protects data in transit, and TEE memory encryption protects processing state. Attestation supplies evidence for the decision; it does not replace encryption.

The protected area normally contains the workload Pod and a limited set of guest helpers. The API server, control plane, host kernel, hypervisor, and other Pods remain outside the confidential boundary and are treated as untrusted. This explicit scope prevents the inaccurate claim that the entire node or every Kubernetes operator is automatically protected.

2. Threat model and trust boundary

A. Threat assumptions

CoCo considers a cloud host administrator or malicious hypervisor that attempts to observe or alter guest execution. It also considers an attacker who controls a node runtime, copies image layers, or abuses Kubernetes debugging and exec operations. The model does not fix application vulnerabilities or malicious code; approved images and policies remain prerequisites.

Typical threats include memory disclosure, image and secret theft from host storage, replacement of the guest boot image, and operational leakage through logs, temporary volumes, or over-privileged debugging.

flowchart LR
    U[Workload Pod] --> G[Guest helper and Kata agent]
    G --> T[TEE boundary]
    T --> H[Encrypted guest memory]
    H -. untrusted .-> HV[Hypervisor]
    HV -. untrusted .-> N[Host kernel and node]
    N -. untrusted .-> CP[Kubernetes control plane]
    CP -. untrusted .-> O[Other pods]

The project places the workload Pod and supporting processes inside the enclave, while the hypervisor, other Pods, and control plane stay outside. This is a compromise between node-centric virtualization, which enlarges the TCB, and container-centric virtualization, which makes normal sharing more complex.

Pod-centric virtualization lets containers in one Pod share a network namespace and volumes without sending their normal traffic outside the enclave. It keeps the guest API relatively small while retaining a useful Kubernetes unit of deployment.

TEE protection is not application protection. A vulnerable application, a malicious image, a weak RBAC policy, or an unsafe network path remains dangerous. Image analysis, application security, Kubernetes authorization, and network policy must therefore be layered with CoCo.

Area Trust boundary Main responsibility
Confidential area Workload Pod and limited guest helpers Execute the application and use secrets
Verification layer TEE measurements and attestation agent Produce execution evidence
Key-release layer KBS, attestation service, and policy store Verify evidence and decide release
Platform area Hypervisor, host, and other Pods Provide resources; untrusted by default
Control area API server, CI/CD, registry, and operators Approve, audit, and change policy

3. Components and processing flow

A. Architecture

sequenceDiagram
    participant Dev as Developer or CI
    participant K8s as Kubernetes API
    participant Kata as Kata runtime and Pod VM
    participant TEE as TEE guest
    participant AS as Attestation service
    participant KBS as Key Broker Service
    participant Reg as Registry
    Dev->>K8s: Deploy Pod with RuntimeClass
    K8s->>Kata: Start confidential Pod VM
    TEE->>AS: Send hardware and guest evidence
    AS-->>KBS: Return appraisal result
    KBS->>TEE: Release image key and secrets
    TEE->>Reg: Pull encrypted or signed image
    TEE-->>K8s: Run workload and report status

Kata Containers runs a Pod in a lightweight VM instead of relying only on the shared host kernel. CoCo adds a hardware TEE and guest components such as a confidential data hub and attestation agent. The Kata Agent manages container lifecycle inside the guest, so its policy and image must be part of the measured trust chain.

The attestation agent produces evidence about the hardware and guest state. Evidence can include the TEE type, guest image digest, initialization data, and runtime configuration. A verifier checks hardware signatures, certificate chains, allowed measurements, and the accepted TCB security level.

In a Trustee-style deployment, the Attestation Service verifies evidence and the Key Broker Service decides which secret a workload may receive. The Confidential Data Hub brokers guest requests so that the application does not need to implement the KBS protocol. Decision and execution are separated, but policy versions and decision logs must be traceable.

B. Remote attestation and conditional release

The normal sequence starts with a challenge containing a nonce. The TEE creates signed evidence for the current environment. The Attestation Service validates the signature and certificate chain, compares measurements with reference values, and evaluates the appraisal policy. If the result passes, the KBS releases the requested image key or secret; otherwise the encrypted image cannot start.

The policy should not say merely “any TEE is allowed.” It can bind a key to the TEE type, guest digest, Kata Agent policy, image digest, namespace, workload identity, purpose, and expiry. A model key, for example, can require both an approved GPU TEE and an exact image digest.

Attestation is not a permanent certificate. Reboots, image changes, TCB patches, key expiry, and policy updates require re-evaluation. Long-running services need session-key or lease renewal. Evidence and logs should contain only the measurements and policy results needed for verification, not personal data or plaintext secrets.

C. Image and secret protection

Image signatures verify provenance and integrity, but do not prevent a registry or host administrator from reading the image. Sensitive model weights and business data need encrypted layers whose decryption keys are released only after successful attestation.

With guest-pull, the host runtime does not unpack plaintext layers. The guest proves its state, receives a key from the KBS, pulls the image, and decrypts and verifies it inside the confidential boundary. Registry credentials must be scoped and kept within the same trust model, or the host can still observe them.

Kubernetes Secrets injected into environment variables may leak through logs, core dumps, or debugging tools. A safer design binds KBS resource policy to workload identity and delivers short-lived credentials only when needed. Data classification must determine whether the application may retain a secret in memory.

D. Kubernetes integration

CoCo uses RuntimeClass to select a protected execution profile. Ordinary Pods can use the default runtime, while sensitive Pods select a class such as kata-qemu-snp or kata-qemu-tdx. Scheduling must account for TEE-capable node labels and resources; otherwise a workload may fail or run with an unintended profile.

Admission policy should check the permitted image, required RuntimeClass, debugging restrictions, node labels, namespace, and service account. All containers and init containers in the Pod must be reviewed together. Exceptions need an owner, reason, duration, and approval record.

4. Deployment choices and comparison

A. Local Pod VM and Peer Pods

With a local Pod VM, the worker node creates the Kata VM and requires suitable TEE hardware or virtualization. It provides direct control but adds responsibility for firmware, nested virtualization, device pass-through, and node replacement.

With Peer Pods, the Cloud API Adaptor calls a cloud API to create a confidential VM outside the ordinary worker node. This reduces the need for TEE bare-metal workers and can use a provider’s confidential-VM service. It adds API latency, network and storage dependencies, external VM cost, and a new failure domain.

Item Local Pod VM Peer Pods
VM location Kubernetes worker node External VM created through a cloud API
Strength Direct node and resource control Uses provider TEE on an existing cluster
Main burden TEE nodes, firmware, and devices API latency, networking, and VM cost
Fit Controlled on-premises or edge Public cloud and multi-tenant platforms
Common requirement Attestation, key release, and image policy remain necessary Attestation, key release, and image policy remain necessary

B. Ordinary containers, Kata, and CoCo

Ordinary containers have high density and fast startup but do not protect data from a host administrator. Kata adds VM isolation, but image confidentiality and remote attestation are not automatically part of every deployment. CoCo adds hardware-backed measurement and conditional secret delivery to the Kata boundary.

The trade-off is extra boot, attestation, key-exchange, and operational cost. GPU, CSI storage, high-performance networking, and debugging tools may conflict with the TEE boundary. A risk-based platform should operate ordinary, Kata, and CoCo profiles together instead of moving every workload into the most expensive profile.

5. Application cases

For a joint financial-feature analysis, each institution can encrypt its inputs and deploy an approved analysis image inside CoCo. The KBS releases a decryption key only when TEE measurements, image digest, and analysis purpose match. Output limits and re-identification controls still apply because a TEE does not remove privacy risk from results.

For a manufacturing AI service using public-cloud GPUs, model weights and input data can be encrypted. A composite attestation policy can release the model key only when CPU and GPU measurements are approved. GPU pass-through, drivers, CUDA, and the model loader then become part of the support matrix and reproducible build process.

For a multi-tenant SaaS analyzer, a tenant key can be bound to tenant identity, processing purpose, and a short lease. Ordinary exec access is denied, while an approved diagnostic image and a masked channel support troubleshooting. CoCo reduces infrastructure trust but does not replace application authorization or tenant isolation.

6. Advanced section — maturity and answer strategy

CoCo is an open-source effort to abstract hardware-specific TEE implementations into a Kubernetes workload model. Actual support varies by CPU, GPU, cloud provider, Kata runtime, Trustee configuration, CSI, and network. Official release notes and hardware matrices must therefore be checked before production use.

In a professional-essay answer, begin with the gap between encryption at rest or in transit and protection of data in use. Present the Pod-centric boundary, then draw the flow from Kata runtime to attestation, KBS, and encrypted image. Compare ordinary containers, Kata, and CoCo by confidentiality, integrity, performance, and operational difficulty. Conclude with key policy, supply chain, observability, exceptions, and performance validation.

Avoid claiming that a TEE stops every attack. It provides a powerful hardware boundary against host inspection, but does not automatically solve application flaws, malicious images, weak key policy, unsafe logs, or denial of service. The guarantee exists only when hardware, guest image, runtime policy, image supply chain, and KBS policy form a verifiable chain.

7. Considerations and implications

A. Minimize the TCB and measurement scope

Minimize the guest kernel, Kata Agent, initialization data, drivers, and GPU components included in the TCB. Manage each version and build hash. Use security review and rollback for reference-value changes, with separate emergency-patch and normal-release policies.

B. Separate key management and policy

The KBS is a policy enforcement point, not just a secret vault. Bind keys to image digest, workload identity, purpose, environment, and expiry, and design a key hierarchy that prevents administrators from seeing plaintext. Separate policy authors from key operators and log decisions without placing secrets in the logs.

C. Protect the supply chain

Build guest and container images reproducibly and connect SBOM, signatures, vulnerability checks, and provenance to CI/CD. Signatures provide integrity and provenance, not confidentiality, so encrypted images may still be required. A fail-closed policy should refuse keys when the approved digest and deployment approval do not match.

D. Design performance and failure modes

Attestation and key exchange add Pod-start latency. Include boot time, registry round trips, KBS failure, and TEE capacity in SLOs. Decide in advance whether existing Pods may continue while new Pods fail closed, and whether any narrowly scoped fail-open behavior is acceptable.

E. Preserve observability and safe debugging

Record attestation result, policy version, image digest, runtime class, and key-release result without sensitive payloads. Disable memory dumps and remote exec by default. Provide a separate non-confidential reproduction environment and an approved diagnostic image.

F. Clarify regulatory responsibility

CoCo strengthens technical protection for personal, financial, and medical data but does not replace lawful purpose, minimization, or retention rules. Contracts and operating procedures should assign ownership for attestation policy, reference values, hardware, and cloud-provider responsibilities.

In one line: Confidential Containers combines Kata-based Pod isolation, hardware TEEs, remote attestation, and conditional key release to protect Kubernetes data in use and workload code from untrusted infrastructure.

References