How to Detect Runtime Threats with Falco on Kubernetes (2026)

Oct 20, 2026 · 15 min · Ajay Kumar

This guide is for platform and security engineers who want real runtime threat detection on a Kubernetes cluster without buying a platform first. By the end you will have Falco running on every node with the modern eBPF driver, the default ruleset plus your own custom rules, alerts flowing through Falcosidekick to Slack and a web UI, a tuned noise level, an optional automated response with Falco Talon, and metrics that tell you when Falco itself is blind. Everything here was checked against Falco 0.45.1 (released 19 October 2026), the falco Helm chart 9.2.0, falco-rules 5.2.0, Falcosidekick chart 0.14 and Falco Talon 0.3.0. You need a test cluster with Linux nodes (x86_64 or ARM64), kubectl, Helm 3, and permission to run a privileged or capability-elevated DaemonSet. If you want the background on why detection is not prevention, read the companion piece on container runtime security in 2026 first.

Step 1: Check that your nodes can run the modern eBPF driver

Falco 0.44 removed the legacy eBPF probe, along with the gVisor engine and gRPC output, so you now choose between two drivers: the modern eBPF probe (bundled in the Falco binary, no compilation, the default) and the kernel module. The driver docs say the modern probe needs BPF ring buffer support and a kernel that exposes BTF, which in practice usually means kernel 5.8 or newer.

Check one node from each node pool. kubectl debug mounts the host filesystem at /host:

# Pick a node and inspect its kernel and BTF availability (busybox 1.37)
NODE=$(kubectl get nodes -o jsonpath='{.items[0].metadata.name}')
kubectl debug node/"$NODE" -it --image=busybox:1.37 -- \
  sh -c 'uname -r; ls -l /host/sys/kernel/btf/vmlinux'

Expected output on a supported node looks like this:

6.8.0-1015-gcp
-r--r--r--    1 root     root       5xxxxxx Oct 20 09:12 /host/sys/kernel/btf/vmlinux

If vmlinux is missing, the modern probe will not start on that node. Upgrade the node image, or plan to use driver.kind=kmod, which needs a fully privileged container and a prebuilt or compiled module for that exact kernel. Delete the debug pod afterwards (kubectl get pods -A | grep node-debugger).

Step 2: Install Falco 0.45 with Helm

Since the 0.44 cycle the chart source lives in the main falcosecurity/falco repository, but it is still published to the same Helm repository. The Falco docs now call the Falco Operator the recommended deployment method, but the operator is still at v0.5.0 release candidates, so I use the chart, which remains fully supported.

helm repo add falcosecurity https://falcosecurity.github.io/charts
helm repo update
helm search repo falcosecurity/falco --versions | head -3
NAME                 CHART VERSION  APP VERSION  DESCRIPTION
falcosecurity/falco  9.2.0          0.45.0       Falco
falcosecurity/falco  9.1.0          0.44.1       Falco

Start with a minimal values file. leastPrivileged: true swaps privileged: true for four capabilities (BPF, SYS_RESOURCE, PERFMON, SYS_PTRACE), which I confirmed by rendering the chart. Chart 9.2.0 ships Falco 0.45.0, so I pin the image to 0.45.1 for its fixes, including a crash when Prometheus scraped metrics mid-event.

# falco-values.yaml (chart falcosecurity/falco 9.2.0)
image:
  tag: "0.45.1"
driver:
  kind: modern_ebpf          # options in 0.45: modern_ebpf, kmod, auto
  modernEbpf:
    leastPrivileged: true    # capabilities instead of a privileged container
tty: true                    # flush logs line by line so kubectl logs is readable
metrics:
  enabled: true              # exposes /metrics on :8765, see Step 9
falco:
  json_output: true          # required by Falcosidekick; easier to parse with jq
  priority: notice           # drop informational and debug alerts
helm install falco falcosecurity/falco \
  --version 9.2.0 \
  --namespace falco --create-namespace \
  -f falco-values.yaml

kubectl -n falco rollout status ds/falco --timeout=180s

Each Falco pod runs two falcoctl containers: an init container that installs the latest compatible falco-rules artifact, and a sidecar that checks for updates every 168 hours (the chart default). Those need egress to falcosecurity.github.io and ghcr.io.

Step 3: Verify that Falco is detecting events

First confirm the driver loaded:

kubectl -n falco logs ds/falco -c falco | grep -E "modern BPF|Loading rules|Enabled event sources"
Opening 'syscall' source with modern BPF probe.

You should also see lines listing the rules files that loaded. Now trigger a stable rule on purpose. Terminal shell in container fires when a shell starts with a TTY attached, which is exactly what kubectl exec -it does:

kubectl create deployment web --image=nginx:1.29
kubectl rollout status deploy/web
kubectl exec -it deploy/web -- sh -c 'id; exit'

kubectl -n falco logs ds/falco -c falco --since=2m | grep '^{' | jq -r '.priority + " " + .rule + " | " + .output_fields["k8s.pod.name"]'
Notice Terminal shell in container | web-6d4cf56db6-x8k2p

The JSON line also carries output_fields such as proc.cmdline, user.name, container.image.repository and k8s.ns.name. The namespace and pod fields come from the container plugin, which reads them from the container runtime socket. kubectl logs ds/falco only reads one pod, so exec into a pod scheduled on the same node or use -l app.kubernetes.io/name=falco with --prefix to see all of them.

For a broader smoke test, run Falco's own event-generator (v0.13.0). It performs suspicious actions for real, so only do this in a test cluster:

kubectl create namespace falco-eg
kubectl -n falco-eg run event-generator --rm -it --restart=Never \
  --image=falcosecurity/event-generator:0.13.0 -- run syscall

Step 4: Understand and extend the default rules

The falcosecurity/rules repository splits rules by maturity. In the falco-rules 5.2.0 release, falco_rules.yaml has 25 stable rules, falco-incubating_rules.yaml has 31 and falco-sandbox_rules.yaml has 39 (I counted the - rule: entries at that tag). Only the stable file loads by default. It covers shells in containers, sensitive file reads, API server contact from pods, release_agent escapes, memfd_create fileless execution, kernel module loads and a few more.

To add the incubating rules, tell falcoctl to fetch the artifact and tell Falco to load the file. Order matters: your custom files in rules.d must load last so they can override earlier definitions.

# add to falco-values.yaml
falcoctl:
  config:
    artifact:
      install:
        refs: [falco-rules:5, falco-incubating-rules:6]
      follow:
        refs: [falco-rules:5, falco-incubating-rules:6]
falco:
  rules_files:
    - /etc/falco/falco_rules.yaml
    - /etc/falco/falco-incubating_rules.yaml
    - /etc/falco/falco_rules.local.yaml
    - /etc/falco/rules.d

Expect more alerts. Incubating rules are useful but less tuned, which is the point of Step 7.

Step 5: Write custom Falco rules

A Falco rule is a YAML object with a condition written in Falco's filter language, an output template, a priority and optional tags and exceptions. Lists and macros keep conditions readable. The chart's customRules value writes files into /etc/falco/rules.d, which loads after the defaults.

This rule catches package managers running inside pods in production namespaces, a common sign of someone installing tools after getting in. It reuses the spawned_process and container macros from the default ruleset:

# add to falco-values.yaml
customRules:
  custom-rules.yaml: |-
    - list: prod_namespaces
      items: [payments, checkout]

    - list: package_mgmt_binaries
      items: [apt, apt-get, dpkg, apk, dnf, yum, rpm, pip, pip3, npm, gem]

    - rule: Package manager run in production pod
      desc: A package manager ran inside a container in a production namespace.
      condition: >
        spawned_process
        and container
        and k8s.ns.name in (prod_namespaces)
        and proc.name in (package_mgmt_binaries)
      output: >
        Package manager in prod pod | ns=%k8s.ns.name pod=%k8s.pod.name
        image=%container.image.repository:%container.image.tag
        cmd=%proc.cmdline user=%user.name parent=%proc.pname
      priority: WARNING
      exceptions:
        - name: approved_migrators
          fields: [k8s.ns.name, container.image.repository]
          values:
            - [payments, ghcr.io/acme/payments-migrator]
      tags: [custom, container, mitre_persistence]

The exceptions block follows the documented syntax: with two fields and no comps, each tuple is compared with =. Exceptions beat bolting and not (...) clauses onto the condition because they stay readable and can be appended to later.

Apply and validate:

helm upgrade falco falcosecurity/falco --version 9.2.0 -n falco -f falco-values.yaml
kubectl -n falco rollout status ds/falco

# Validate the rendered rules file inside a running pod
kubectl -n falco exec ds/falco -c falco -- \
  falco --validate /etc/falco/rules.d/custom-rules.yaml

Two 2026 changes affect rule authors. Falco 0.44 added validation for unknown keys in rules files, so typos that used to pass silently now get flagged. Falco 0.45 evaluates conditions on raw bytes, and the release notes say rules that matched the Unicode replacement character with non-regex operators must be reviewed.

Step 6: Send alerts to Slack and a UI with Falcosidekick

Falco writes alerts to stdout, syslog, files or an HTTP endpoint. Falcosidekick takes the HTTP stream and fans it out to dozens of destinations (Slack, Loki, Elasticsearch, Alertmanager, webhooks and more) with a minimum priority per output. Enabling it in the chart also turns on json_output and points http_output at http://falco-falcosidekick:2801.

Keep the webhook out of your values file. The Falcosidekick chart reads environment variables from an existing secret, using upper-case names such as SLACK_WEBHOOKURL:

kubectl -n falco create secret generic falcosidekick-outputs \
  --from-literal=SLACK_WEBHOOKURL='https://hooks.slack.com/services/T000/B000/XXXX'
# add to falco-values.yaml
falcosidekick:
  enabled: true
  config:
    existingSecret: falcosidekick-outputs
    slack:
      minimumpriority: "warning"     # Notice alerts stay in the UI, not Slack
  webui:
    enabled: true
    user: "admin:REPLACE_ME"         # the chart default is admin:admin
helm upgrade falco falcosecurity/falco --version 9.2.0 -n falco -f falco-values.yaml
kubectl -n falco port-forward svc/falco-falcosidekick-ui 2802:2802
# open http://localhost:2802 and log in

Change the UI password. The chart default is admin:admin, and the UI shows every alert, including command lines that may contain secrets. Note that the subchart's default Falcosidekick image is 2.32.0 while the project has released 2.35.0; set falcosidekick.image.tag if you need a newer output.

Step 7: Tune noise before anyone stops reading alerts

An untuned Falco on a busy cluster produces alerts nobody reads, and an ignored detector is worse than none because it creates false comfort. Find the loudest rules first:

kubectl -n falco logs -l app.kubernetes.io/name=falco -c falco --since=24h --tail=-1 \
  | grep '^{' | jq -r '.rule' | sort | uniq -c | sort -rn | head -10

Then tune with the least invasive tool that works:

  1. Use the rule's own tuning macro. Many default rules include a user_expected_* macro that defaults to never_true. Override it instead of editing the rule.
  2. Add exceptions to rules you own, as in Step 5.
  3. Disable a rule by name or tag in falco.rules, which applies after all files load.
# add to customRules -> custom-rules.yaml
    - macro: user_expected_terminal_shell_in_container_conditions
      condition: (k8s.ns.name = "debug-tools")
      override:
        condition: replace
# add to falco-values.yaml under falco:
  rules:
    - disable:
        rule: "Contact K8S API Server From Container"

The override syntax replaced the old append: true, which the overriding docs say will be removed in Falco 1.0.0. Do not disable a rule because it fired once during a deploy. Disable it when you have decided you will never act on it.

Step 8: Automate a response with Falco Talon

The Falco pipeline in this guide JSON over HTTP :2801 Node kernel execve, openat, connect Modern eBPF probe CO-RE, BTF, ring buffer Falco engine rules + exceptions Falcosidekick fan-out by priority :2803 Slack, Loki, SIEM per-output minimum Falco Talon match, dry_run first Kubernetes API label, netpol, kill pod Falco's own /metrics on :8765: alert on falcosecurity_scap_n_drops_total before trusting silence. Talon has no default target: set falcosidekick.config.talon.address or no actions fire.
Components and ports as rendered by the falco 9.2.0 chart with Falcosidekick and Talon enabled. Falco only detects; every action happens downstream, after the alert.

Falco Talon receives events from Falcosidekick and runs actions through "actionners" such as kubernetes:label, kubernetes:networkpolicy and kubernetes:terminate. Its latest release is v0.3.0 from February 2025, so treat it as young software and start every rule in dry-run mode. The Falco chart deploys it as a subchart (falco-talon 0.4.x) when responseActions.enabled is true, and the subchart predefines two actions, Terminate Pod and Label Pod as Suspicious (which sets analysis/status: suspicious).

One trap I found by rendering the chart: enabling Talon does not wire Falcosidekick to it. TALON_ADDRESS renders empty unless you set it.

# add to falco-values.yaml
responseActions:
  enabled: true
falcosidekick:
  config:
    talon:
      address: "http://falco-talon:2803"
      minimumpriority: "warning"
falco-talon:
  config:
    rulesOverride: |
      - rule: Label pods that run package managers in prod
        match:
          rules:
            - Package manager run in production pod
        dry_run: true                  # log what would happen; flip to false later
        actions:
          - action: Label Pod as Suspicious

The rule keys (match.rules, match.priority, match.output_fields, dry_run, continue) are described in the Talon rules docs. Once you trust it, pair the label with a NetworkPolicy that isolates analysis/status=suspicious pods. That preserves the pod for forensics, where terminating it destroys the evidence.

Step 9: Monitor Falco itself

Falco being quiet is only good news if Falco is healthy. With metrics.enabled: true, the chart turns on Prometheus metrics at /metrics on port 8765 alongside /healthz. The metrics docs list falcosecurity_scap_n_drops_total for kernel-side drops and falcosecurity_falco_rules_matches_total (labelled by rule and priority) for alert counts.

kubectl -n falco port-forward ds/falco 8765:8765 &
curl -s localhost:8765/metrics | grep -E '^falcosecurity_scap_n_drops_total|^falcosecurity_falco_rules_matches_total' | head
# PrometheusRule fragment: page when any node drops events for 10 minutes
- alert: FalcoDroppingEvents
  expr: rate(falcosecurity_scap_n_drops_total[5m]) > 0
  for: 10m
  labels:
    severity: warning
  annotations:
    summary: "Falco on {{ $labels.instance }} is dropping kernel events"

Set serviceMonitor.create: true if you run the Prometheus Operator. If drops persist, the dropping events guide suggests driver.modernEbpf.bufSizePreset of 6 or 7 with cpusForEachBuffer of 4 or 6 (chart defaults are 4 and 2), at the cost of more preallocated memory.

Troubleshooting

Symptom Likely cause Fix
Falco pod in CrashLoopBackOff right after Opening 'syscall' source with modern BPF probe. Node kernel lacks BTF or BPF ring buffer Check /sys/kernel/btf/vmlinux (Step 1); upgrade the node image or set driver.kind=kmod
Upgrade fails with an unknown driver kind or missing probe Values still set driver.kind=ebpf; legacy probe removed in 0.44 Switch to modern_ebpf or kmod
operation not permitted loading BPF with leastPrivileged: true Kernel or AppArmor policy blocks the capability-only mode Set driver.modernEbpf.leastPrivileged: false to run privileged, then investigate
Init container falcoctl-artifact-install fails No egress to falcosecurity.github.io or ghcr.io (air-gapped) Mirror the artifacts, or set falcoctl.artifact.install.enabled=false and follow.enabled=false to use the image's embedded rules
k8s.ns.name and k8s.pod.name show <NA> Container plugin cannot reach the runtime socket, or lookup lag on very short-lived processes Check collectors.containerEngine socket paths; chart 9.2.0 mounts socket directories so runtime restarts no longer break it
Falco fails to load rules after upgrading to 0.44 or 0.45 Unknown-key validation added in 0.44; raw-byte matching changed in 0.45 Run falco --validate on each file and read the release notes' breaking changes
Falcosidekick UI is empty Falcosidekick deployed separately without json_output and http_output enabled Enable it through the Falco chart, or set both options yourself
Talon receives nothing or does nothing falcosidekick.config.talon.address unset, rule still dry_run: true, or RBAC missing Set the address, check Talon logs, flip dry_run deliberately
Alerts Falco internal: syscall event drop or rising drop counters Ring buffer too small for the node's syscall rate Raise bufSizePreset and cpusForEachBuffer; trim noisy rules
Hundreds of Terminal shell in container alerts Engineers debugging with kubectl exec Override user_expected_terminal_shell_in_container_conditions for a debug namespace, and correlate with audit logs

FAQ

Does Falco block attacks or only detect them?

Falco only detects. It observes system calls after they happen and emits alerts. Blocking requires another layer: Falco Talon can react after an alert (label, isolate or kill a pod), while tools such as Tetragon or KubeArmor can deny operations inside the kernel.

Should I use the modern eBPF driver or the kernel module?

Use modern eBPF wherever the kernel exposes BTF. It ships inside the Falco binary, needs no compilation, and can run with capabilities instead of full privilege. The kernel module is the fallback for older kernels and always needs a privileged container.

How much CPU and memory does Falco need?

The chart defaults to requests of 100m CPU and 512Mi memory with limits of 1 CPU and 1Gi per node. Real usage depends on syscall rate and how many rules you load, so watch the drop counters and resource metrics for a week before tightening limits.

Does Falco work on serverless Kubernetes nodes?

Only where you can run a DaemonSet with the required capabilities on nodes you can see. Offerings that hide the node or forbid elevated DaemonSets do not support the syscall driver; check your provider's documentation for a partner or managed option.

In production, the first thing I check on a Falco install is not the rules. It is whether every node is actually running the probe and whether falcosecurity_scap_n_drops_total is flat, because a detector with gaps is worse than a known blind spot. Then I look at which five rules generate most alerts and decide, rule by rule, whether anyone will act on them. Custom rules for the invariants you care about (no package managers in prod, no shells in payment pods, nothing writing under /proc/sys) are worth more than the incubating set. For the details, keep the Falco documentation, the chart README and the Falcosidekick outputs list open. If you are hardening the rest of the cluster too, Kubernetes network policies pair naturally with Talon's isolation actions.

Related: Container vs microVM vs gVisor: Choosing Agent Isolation, It's 2 AM. Do You Know What Your AI Agent Is Doing? and Observability in 2026: OpenTelemetry Graduated, the Collector Is Still v0.162, eBPF Instrumentation Is v0.14, and Datadog Bills $1.12B a Quarter.

I'm Ajay Kumar — I build and operate PandaStack, an open-source Firecracker microVM cloud for AI agents, and write about the infrastructure and tooling around it.

Need this kind of infrastructure work? See what I do or email hello@ajayk.sh.


Related