Learn Cilium - Runtime Security with Tetragon

#Introduction

Cilium protects the network between workloads. But what about what happens inside a pod — which processes are run, which files are accessed, which commands an attacker who already got in executes? Episode 16 answers with Tetragon: an eBPF-based runtime security project in the same family as Cilium.

Tetragon monitors syscalls directly in the kernel with eBPF — without an agent that modifies applications. It can detect shell execution inside a container, access to sensitive files, or suspicious behavioral patterns, then record them as security events. You will install Tetragon, write a TracingPolicy, and see how it complements Cilium.

#What Is Tetragon

Tetragon is a CNCF (sandbox) project that leverages eBPF for runtime security observability and enforcement. The difference from Cilium: Cilium sees traffic between workloads, Tetragon sees behavior inside workloads — process execution, file access, per-process network connections, and container lifecycle. The two complement each other: one protects the network perimeter, the other watches what is inside.

It is important to understand that Tetragon is not an antivirus. It does not detect based on signatures; it monitors behavior based on the policies you write. Its strength lies in precision: you define exactly what is watched, rather than relying on a signature database that can go stale.

Tetragon uses a CRD named TracingPolicy to define what is monitored. A policy can be observability (recording) or enforcement (blocking). All events are emitted to user-space without modifying the target pod — exactly the Cilium philosophy.

#Installing Tetragon

Tetragon is installed separately from Cilium, using the same Helm repository:

helm repo add tetragon https://helm.cilium.io/
helm repo update
helm install tetragon tetragon/tetragon -n kube-system

helm install tetragon tetragon/tetragon -n kube-system installs Tetragon as a DaemonSet in kube-system. Verify that the agent is running:

kubectl get pods -n kube-system -l app.kubernetes.io/name=tetragon

kubectl get pods -n kube-system -l app.kubernetes.io/name=tetragon shows one agent pod per node. Tetragon requires no changes to your workloads — it only attaches eBPF programs to the kernel.

#TracingPolicy: Process Execution and File Access

TracingPolicy is the heart of Tetragon. The simplest example: monitoring the execution of specific processes in all containers:

apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: pantau-exec
spec:
  kprobes:
    - call: "sys_execve"
      syscall: true
      args:
        - index: 0
          type: "string"
      selectors:
        - matchArgs:
            - index: 0
              operator: "In"
              values:
                - "/bin/bash"
                - "/bin/sh"

call: "sys_execve" targets the program-execution syscall, and the selector limits it to suspicious arguments such as /bin/sh. When this pattern is detected, Tetragon produces a security event. The same pattern can be used with the openat syscall to monitor access to sensitive files.

#Detecting Suspicious Behavior

With the TracingPolicy applied, Tetragon starts emitting events. Watch events directly via the CLI:

kubectl logs -n kube-system -l app.kubernetes.io/name=tetragon -f

kubectl logs -n kube-system -l app.kubernetes.io/name=tetragon -f displays events in real time in JSON format. To test it, open an interactive shell in any pod — an exec event will appear recording the /bin/bash process along with its originating container and pod.

Common detection patterns:

• Shell execution inside a container that should be headless.
• Access to credential files such as /etc/shadow or secret keys.
• Outbound connections to suspicious addresses from a specific process.
• Invocations of binaries that are not registered in the image.

All of these patterns can be forwarded to a SIEM (for example Elasticsearch or Loki) for further analysis — an approach we will consider in the production architecture in episode 21.

#Cilium and Tetragon Integration

When used together, Cilium and Tetragon form a complete security posture:

• Cilium controls who may communicate with whom (network policy).
• Tetragon watches what happens inside the workloads that are allowed to communicate.
• Both use eBPF, so they do not add heavy agents to each pod.

This combination closes a typical scenario: a network policy allows pod A to reach pod B, but an attacker already inside pod B could execute code unnoticed — until Tetragon records the suspicious execution. With alerting integrated, the security team gets warned far earlier than when waiting for the damage to happen.

#Closing

Key takeaways:

• Tetragon monitors the runtime (processes, files, connections) with eBPF without changing workloads.
• A TracingPolicy defines what is monitored and whether it will be blocked.
• sys_execve and openat are common hooks for monitoring execution and file access.
• Tetragon events can be watched in real time via the agent logs.
• Cilium secures the network; Tetragon secures behavior inside workloads.
• Start in observability mode before enabling enforcement.

In the next episode 17, we will cover multi-cluster and ClusterMesh — connecting multiple clusters, cross-cluster service discovery, the MCS API that became stable in version 1.20, and failover and service portability scenarios between clusters. This takes your architecture from one cluster to many clusters.