Configure Falco runtime security for Kubernetes threat detection with eBPF monitoring

Update system packages

Start by updating your package manager to ensure compatibility with Falco installation dependencies.

sudo apt update && sudo apt upgrade -y
sudo apt install -y curl gnupg2 software-properties-common

sudo dnf update -y
sudo dnf install -y curl gnupg2 yum-utils

Add Falco repository

Add the official Falco repository to access the latest stable packages and security updates.

curl -fsSL https://falco.org/repo/falcosecurity-packages.asc | sudo gpg --dearmor -o /usr/share/keyrings/falco-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/falco-archive-keyring.gpg] https://download.falco.org/packages/deb stable main" | sudo tee /etc/apt/sources.list.d/falcosecurity.list
sudo apt update

sudo rpm --import https://falco.org/repo/falcosecurity-packages.asc
sudo curl -s -o /etc/yum.repos.d/falcosecurity.repo https://falco.org/repo/falcosecurity-rpm.repo

Install Falco with eBPF support

Install Falco with eBPF driver support, which provides better performance and compatibility than kernel modules.

sudo apt install -y falco
sudo apt install -y linux-headers-$(uname -r)

sudo dnf install -y falco
sudo dnf install -y kernel-devel-$(uname -r) kernel-headers-$(uname -r)

Configure Falco for eBPF

Configure Falco to use the eBPF driver instead of kernel modules for better security and compatibility.

engine:
  kind: ebpf
  ebpf:
    host_root: /host
    buf_size_preset: 4
    drop_failed_exit: false

# Enable JSON output for better parsing
json_output: true
json_include_output_property: true
json_include_tags_property: true

# Configure log output
log_stderr: true
log_syslog: false
log_level: info

# Enable metrics
metrics:
  enabled: true
  interval: 1h
  output_rule: true
  rules_counters_enabled: true
  resource_utilization_enabled: true
  state_counters_enabled: true
  kernel_event_counters_enabled: true
  libbpf_stats_enabled: true
  convert_memory_to_mb: true
  include_empty_values: false

# Web server for health checks and metrics
webserver:
  enabled: true
  listen_port: 8765
  k8s_healthz_endpoint: /healthz
  ssl_enabled: false
  prometheus_metrics_enabled: true

Install Falco on Kubernetes with Helm

Deploy Falco as a DaemonSet on your Kubernetes cluster to monitor all nodes for security events.

helm repo add falcosecurity https://falcosecurity.github.io/charts
helm repo update

# Create namespace for Falco
kubectl create namespace falco

# Install Falco with eBPF and Prometheus integration
helm install falco falcosecurity/falco \
  --namespace falco \
  --set driver.kind=ebpf \
  --set collectors.enabled=true \
  --set falco.webserver.enabled=true \
  --set falco.metrics.enabled=true \
  --set serviceMonitor.enabled=true \
  --set falco.json_output=true \
  --set falco.json_include_output_property=true \
  --set falco.log_level=info

Configure custom Falco rules

Create custom security rules tailored to your Kubernetes environment and application requirements.

# Custom Kubernetes security rules
- rule: Detect privilege escalation
  desc: Detect attempts to escalate privileges in containers
  condition: >
    spawned_process and container and
    (proc.name in (sudo, su) or
     proc.args contains "--privileged" or
     proc.args contains "--cap-add")
  output: >
    Privilege escalation attempt detected (user=%user.name
    container=%container.name image=%container.image.repository
    command=%proc.cmdline)
  priority: HIGH
  tags: [kubernetes, privilege_escalation, mitre_privilege_escalation]

- rule: Suspicious file access in container
  desc: Detect access to sensitive files in containers
  condition: >
    open_read and container and
    (fd.name startswith "/etc/passwd" or
     fd.name startswith "/etc/shadow" or
     fd.name startswith "/root/.ssh" or
     fd.name startswith "/home/*/.ssh")
  output: >
    Suspicious file access (user=%user.name container=%container.name
    image=%container.image.repository file=%fd.name command=%proc.cmdline)
  priority: HIGH
  tags: [kubernetes, file_access, credential_access]

- rule: Container escape attempt
  desc: Detect potential container escape attempts
  condition: >
    spawned_process and container and
    (proc.name in (docker, runc, ctr, crictl) or
     proc.args contains "nsenter" or
     proc.args contains "unshare" or
     proc.args contains "chroot")
  output: >
    Container escape attempt detected (user=%user.name
    container=%container.name command=%proc.cmdline)
  priority: CRITICAL
  tags: [kubernetes, container_escape, mitre_escape]

- rule: Unauthorized network connection
  desc: Detect unexpected outbound network connections
  condition: >
    outbound and container and
    not fd.sip in (cluster_ip_range, service_ip_range) and
    not fd.sport in (53, 443, 80) and
    not proc.name in (curl, wget, apt, yum, dnf)
  output: >
    Unauthorized network connection (user=%user.name
    container=%container.name dest=%fd.sip:%fd.sport command=%proc.cmdline)
  priority: MEDIUM
  tags: [kubernetes, network, exfiltration]

- rule: Crypto mining activity
  desc: Detect potential cryptocurrency mining activity
  condition: >
    spawned_process and
    (proc.name in (xmrig, cpuminer, cgminer, bfgminer) or
     proc.args contains "stratum+tcp" or
     proc.args contains "mining" or
     proc.cmdline contains "--donate-level")
  output: >
    Crypto mining activity detected (user=%user.name
    container=%container.name command=%proc.cmdline)
  priority: CRITICAL
  tags: [kubernetes, cryptomining, resource_abuse]

Configure Falco logging output

Set up structured logging to forward security events to your centralized logging system.

apiVersion: v1
kind: ConfigMap
metadata:
  name: falco-config
  namespace: falco
data:
  falco.yaml: |
    rules_file:
      - /etc/falco/falco_rules.yaml
      - /etc/falco/falco_rules.local.yaml
      - /etc/falco/k8s_audit_rules.yaml
    
    json_output: true
    json_include_output_property: true
    json_include_tags_property: true
    
    file_output:
      enabled: true
      keep_alive: false
      filename: "/var/log/falco/events.log"
    
    stdout_output:
      enabled: true
    
    syslog_output:
      enabled: false
    
    program_output:
      enabled: false
      keep_alive: false
      program: "jq '{timestamp: .time, rule: .rule, priority: .priority, container: .output_fields.container_name, image: .output_fields.container_image_repository}' | logger -t falco-json"
    
    http_output:
      enabled: false
      url: "http://your-webhook-endpoint/falco"
      user_agent: "falcosecurity/falco"
      ca_cert: ""
      ca_bundle: ""
      ca_path: ""
      insecure: false
    
    grpc:
      enabled: false
      bind_address: "0.0.0.0:5060"
      threadiness: 0
    
    grpc_output:
      enabled: false

Apply the logging configuration

Update your Falco deployment with the custom logging configuration.

kubectl apply -f falco-logging-config.yaml

# Restart Falco pods to pick up new configuration
kubectl rollout restart daemonset/falco -n falco

# Verify the restart completed
kubectl rollout status daemonset/falco -n falco

Set up Prometheus monitoring

Configure Prometheus to scrape Falco metrics for monitoring security event trends and system performance.

apiVersion: v1
kind: ConfigMap
metadata:
  name: prometheus-falco-config
  namespace: monitoring
data:
  falco-scrape-config.yaml: |
    - job_name: 'falco-metrics'
      kubernetes_sd_configs:
      - role: pod
        namespaces:
          names:
          - falco
      relabel_configs:
      - source_labels: [__meta_kubernetes_pod_label_app_kubernetes_io_name]
        action: keep
        regex: falco
      - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
        action: keep
        regex: true
      - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
        action: replace
        target_label: __metrics_path__
        regex: (.+)
      - source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
        action: replace
        regex: ([^:]+)(?::\d+)?;(\d+)
        replacement: $1:$2
        target_label: __address__
      - action: labelmap
        regex: __meta_kubernetes_pod_label_(.+)
      - source_labels: [__meta_kubernetes_namespace]
        action: replace
        target_label: kubernetes_namespace
      - source_labels: [__meta_kubernetes_pod_name]
        action: replace
        target_label: kubernetes_pod_name
      scrape_interval: 30s
      scrape_timeout: 10s
      metrics_path: /metrics

Create Falco ServiceMonitor for Prometheus

Set up automatic service discovery for Falco metrics if you're using the Prometheus Operator.

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: falco-metrics
  namespace: falco
  labels:
    app.kubernetes.io/name: falco
    prometheus: kube-prometheus
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: falco
  endpoints:
  - port: http-metrics
    interval: 30s
    path: /metrics
    scheme: http
    scrapeTimeout: 10s
  namespaceSelector:
    matchNames:
    - falco

Apply Prometheus monitoring configuration

Deploy the ServiceMonitor to enable automatic metrics collection from Falco instances.

kubectl apply -f falco-servicemonitor.yaml

# Verify ServiceMonitor is created
kubectl get servicemonitor -n falco

# Check if Prometheus is discovering Falco targets
kubectl port-forward -n monitoring svc/prometheus-kube-prometheus-prometheus 9090:9090 &

</code><h2><code>Visit http://localhost:9090/targets to verify Falco endpoints</code></h2></pre>
</div>

<div class="step">
### Configure alerting rules
<p>Create Prometheus alerting rules to notify your team when Falco detects security threats.</p>
<pre class="terminal"><code>apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: falco-security-alerts
  namespace: falco
  labels:
    prometheus: kube-prometheus
    role: alert-rules
spec:
  groups:
  - name: falco.security
    rules:
    - alert: FalcoSecurityThreatHigh
      expr: increase(falco_events_total{priority="Critical"}[5m]) > 0
      for: 0m
      labels:
        severity: critical
        component: falco
      annotations:
        summary: "Critical security threat detected by Falco"
        description: "Falco detected {{ $value }} critical security events in the last 5 minutes. Rule: {{ $labels.rule }}. Pod: {{ $labels.kubernetes_pod_name }}"
    
    - alert: FalcoSecurityThreatMedium
      expr: increase(falco_events_total{priority="High"}[10m]) > 3
      for: 2m
      labels:
        severity: warning
        component: falco
      annotations:
        summary: "Multiple high-priority security threats detected"
        description: "Falco detected {{ $value }} high-priority security events in the last 10 minutes. This may indicate ongoing malicious activity."
    
    - alert: FalcoDown
      expr: up{job="falco-metrics"} == 0
      for: 3m
      labels:
        severity: critical
        component: falco
      annotations:
        summary: "Falco is down"
        description: "Falco has been down for more than 3 minutes. Security monitoring is compromised on {{ $labels.kubernetes_pod_name }}."
    
    - alert: FalcoHighEventRate
      expr: rate(falco_events_total[5m]) > 10
      for: 5m
      labels:
        severity: warning
        component: falco
      annotations:
        summary: "High Falco event rate detected"
        description: "Falco is generating events at a rate of {{ $value }} per second, which may indicate system issues or attacks."
    
    - alert: FalcoDroppedEvents
      expr: increase(falco_kernel_drops_total[5m]) > 0
      for: 2m
      labels:
        severity: warning
        component: falco
      annotations:
        summary: "Falco is dropping kernel events"
        description: "Falco dropped {{ $value }} kernel events in the last 5 minutes, potentially missing security threats."
    
    - alert: FalcoContainerEscape
      expr: increase(falco_events_total{rule=~".<em>container.</em>escape.*"}[1m]) > 0
      for: 0m
      labels:
        severity: critical
        component: falco
      annotations:
        summary: "Container escape attempt detected"
        description: "Potential container escape attempt detected on {{ $labels.kubernetes_pod_name }}. Immediate investigation required."

Apply alerting rules

Deploy the Prometheus alerting rules to monitor Falco security events.

kubectl apply -f falco-alerts.yaml

# Verify the PrometheusRule is created
kubectl get prometheusrule -n falco

# Check if Prometheus loaded the rules
kubectl logs -n monitoring -l app.kubernetes.io/name=prometheus -c prometheus | grep "falco"

Test Falco detection

Trigger test security events to verify that Falco is properly detecting and alerting on suspicious activities.

# Create a test pod with suspicious activity
kubectl run falco-test --image=ubuntu:22.04 --rm -it --restart=Never -- /bin/bash

# Inside the test pod, run commands that should trigger Falco rules:
# Try to read sensitive files
cat /etc/passwd
cat /etc/shadow

# Attempt privilege escalation (this will fail but should be detected)
sudo whoami

# Create network connections
nc -zv google.com 80

# Exit the test pod
exit

View Falco security events

Check that Falco detected the test activities and generated appropriate security alerts.

# View recent Falco logs
kubectl logs -n falco -l app.kubernetes.io/name=falco --tail=50

# Check Falco metrics in Prometheus
kubectl port-forward -n monitoring svc/prometheus-kube-prometheus-prometheus 9090:9090 &

# Query for recent security events
curl -G 'http://localhost:9090/api/v1/query' --data-urlencode 'query=falco_events_total' | jq

# View events by priority
curl -G 'http://localhost:9090/api/v1/query' --data-urlencode 'query=falco_events_total{priority="High"}' | jq

Enable Kubernetes audit log integration

Configure Falco to analyze Kubernetes API audit logs for detecting suspicious administrative activities.

apiVersion: audit.k8s.io/v1
kind: Policy
rules:
- level: Metadata
  namespaces: ["kube-system", "falco"]
  verbs: ["create", "update", "patch", "delete"]
  resources:
  - group: ""
    resources: ["secrets", "configmaps"]
  - group: "rbac.authorization.k8s.io"
    resources: ["clusterroles", "clusterrolebindings"]
  omitStages:
    - RequestReceived

- level: Request
  verbs: ["create", "update", "patch"]
  resources:
  - group: ""
    resources: ["pods/exec", "pods/attach", "pods/portforward"]
  omitStages:
    - RequestReceived

- level: Metadata
  verbs: ["create", "delete"]
  resources:
  - group: ""
    resources: ["pods", "services"]
  - group: "apps"
    resources: ["deployments", "daemonsets"]
  omitStages:
    - RequestReceived

Update kube-apiserver for audit logging

Configure the Kubernetes API server to send audit logs to Falco for analysis.

# Add these flags to kube-apiserver command:
- --audit-log-path=/var/log/audit.log
- --audit-policy-file=/etc/kubernetes/audit-policy.yaml
- --audit-log-maxage=30
- --audit-log-maxbackup=10
- --audit-log-maxsize=100

# Add volume mounts for audit policy
volumeMounts:
- mountPath: /etc/kubernetes/audit-policy.yaml
  name: audit-policy
  readOnly: true
- mountPath: /var/log/audit.log
  name: audit-log
  readOnly: false

volumes:
- name: audit-policy
  hostPath:
    path: /etc/kubernetes/audit-policy.yaml
    type: File
- name: audit-log
  hostPath:
    path: /var/log/audit.log
    type: FileOrCreate

Create custom detection rules for your environment

Add application-specific security rules based on your workload patterns and security requirements.

# Application-specific security rules
apiVersion: v1
kind: ConfigMap
metadata:
  name: custom-falco-rules
  namespace: falco
data:
  custom_rules.yaml: |
    - rule: Suspicious database access
      desc: Detect direct database file access outside normal operations