Learn Kata Containers - AI & Agent Sandboxes

#Introduction

Of all the use cases covered throughout this series, nothing is more relevant to recent years than AI agents. These agents run tools, access files, and execute commands from sources that can't be fully trusted. Every agent is a potential sandbox escape. Kata Containers provides a natural answer: run every agent in its own microVM.

Episode 19 covers this pattern concretely: running LLM inference in a microVM with GPUs via VFIO, creating per-agent and per-session sandboxes, and CI/CD isolation that executes third-party artifacts. This is a topic already discussed at the OpenInfra Summit and adopted by platforms like Northflank.

#Why AI Agents Need a Sandbox

#The Untrustworthy Nature of Agents

Modern AI agents work like this: the model (LLM) decides a step, then executes — running code, calling APIs, reading files, installing packages. The inputs guiding these decisions come from users, from web content, or from tools interacting with the real world. One jailbroken prompt or one malicious document can make an agent run harmful commands.

The risk isn't theoretical — there have already been many incidents where agents executed malicious code hidden in web pages or documents they read. Regular containers give agents access to a shared kernel; one escape puts the whole node at risk.

#Agent Sandbox vs Traditional Sandbox

A traditional sandbox isolates input: run an unknown file and discard the result. An agent is different — it's active and persistent: browsing, interacting, and surviving for a long time. The isolation needed is stricter, and the microVM is the most fitting answer.

The advantages of a microVM for agents:

• A separate kernel: escaping the agent process doesn't break through the host kernel.
• Restricted egress: with the episode 14 policy, an agent can only communicate with allowed endpoints.
• Destroy-and-recreate: a suspicious sandbox can be destroyed completely without a trace.

#Running LLM Inference in a MicroVM

#Inference Workloads

LLM inference is a compute workload that demands GPUs. With Kata, you can run an inference server (for example vLLM or TGI) inside a microVM with a GPU passed through via VFIO — the combination covered in episode 10.

This pattern is attractive for two reasons: inference input is untrustworthy (prompts from users can be dangerous), and the model itself is valuable (intellectual property that must be protected). The microVM isolates both: the model runtime is protected from malicious prompts, and the model is protected from the node.

An inference pod with a GPU:

apiVersion: v1
kind: Pod
metadata:
  name: kata-llm
spec:
  runtimeClassName: kata
  containers:
    - name: inference
      image: vllm/vllm-openai:latest
      command: ["vllm", "serve", "/models/llama", "--port", "8000"]
      resources:
        limits:
          nvidia.com/gpu: 1
          memory: 32Gi

nvidia.com/gpu: 1 gives the GPU to the microVM, and the entire inference server runs inside the guest. Egress policy limits the model's access to allowed endpoints only.

#Per-Agent and Per-Session Sandbox Patterns

#Per-Agent Sandbox

The simplest pattern: each agent gets its own pod/microVM. One agent, one microVM, one lifecycle. When the agent finishes or becomes suspicious, the microVM is destroyed — no state remains.

This pattern is used by platforms like Northflank and was visualized at the OpenInfra Summit: the per-agent sandbox is the most natural isolation unit. Agents never share a kernel or resources with other agents.

#Per-Session Sandbox

A more granular variation: every interaction session gets a new sandbox. A long-running agent can start a new session for every task involving untrusted input — a suspicious session is discarded without affecting the main agent.

The advantage: isolation granularity follows the input's risk level. Trusted input can be processed directly by the agent; suspicious input goes into a new session sandbox.

The practical implementation stays simple — a new pod per session:

kubectl apply -f - <<'EOF'
apiVersion: v1
kind: Pod
metadata:
  name: agent-session-42
  namespace: sandbox
spec:
  runtimeClassName: kata
  containers:
    - name: executor
      image: sandbox-runner:latest
      env:
        - name: SESSION_ID
          value: "42"
EOF

kubectl apply -f - creates a new session sandbox pod. The sandbox namespace is already guaranteed to use Kata by admission policy (episode 15), so there's no risk of running on runc.

#Resources and Expiry

Per-session sandboxes demand disciplined resource management:

• TTL: destroy the sandbox automatically after a set time.
• Quota: limit resources per session (episode 16) so one session can't consume a node.
• Monitoring: monitor per-microVM (episode 16) to detect suspicious behavior.

#CI/CD Isolates

#The CI/CD Problem

CI/CD executes fundamentally untrusted code — artifacts from contributors, dependencies from the internet, third-party build scripts. Running builds in regular containers means every build has access to the shared kernel. One compromised dependency can attack the runner host.

Kata solves this with the same pattern: every build job runs in a microVM. Build pipelines that use Kubernetes (for example with Kubernetes Runners) just mark the pod as sandboxed:

apiVersion: v1
kind: Pod
metadata:
  name: ci-job-1234
spec:
  runtimeClassName: kata
  restartPolicy: Never
  containers:
    - name: build
      image: builder:latest
      command: ["/build.sh"]

restartPolicy: Never — a one-shot job, the microVM is created for this job and destroyed after it finishes. This pattern gives full isolation between builds without sacrificing adequate speed (150-300 ms boot).

#Egress for Builds

Builds need access to registries and repositories — but not the internal network. A strict egress policy (episode 14) limits builds to only the needed endpoints: image registries, package managers, and source repositories. If a suspicious build tries other access, the traffic is blocked.

#Conclusion

What you should take away:

• AI agents execute unknown code — the microVM is the most fitting sandbox.
• LLM inference can run in a microVM with a GPU via VFIO.
• Two inference isolation directions: the runtime from malicious prompts, the model from prying eyes.
• Per-agent and per-session sandbox patterns give granularity matching input risk.
• CI/CD isolates: every build job runs in a microVM with restricted egress.
• TTL, quota, and per-VM monitoring keep this pattern healthy.

In the next episode, episode 20, we'll cover performance & optimization — hugepages, CPU pinning, memory ballooning, boot time optimization with Dragonball/CH, and how to benchmark boot time and per-pod overhead compared to runc. All the concepts from episodes 5 and 6 will be used to make microVMs as efficient as possible.