Firecracker's seccomp-BPF filters explained: locking down the VMM's own syscalls

Most seccomp explainers talk about confining the workload — the container, the guest, the untrusted program. This post is about the twist that makes Firecracker's model interesting: Firecracker points seccomp at itself. The Virtual Machine Monitor — the host-side process that opens /dev/kvm, runs the vCPU loop, serves the API socket, and emulates the guest's virtio devices — installs a strict seccomp-BPF filter on its own threads at startup. The bet is blunt: even if an attacker gets full code execution inside the Firecracker process, the set of host syscalls they can make is tiny — an allowlist of a few dozen — and everything else terminates the offending thread on the spot. I'm Ajay; I built PandaStack, and every sandbox we run leans on exactly this. Here's how the filter actually works.

What seccomp-BPF actually is

Every time a userspace process wants the kernel to do something real — read a file, send a packet, map memory, spawn a child — it makes a system call. The syscall interface is the one door between a process and the kernel, and it is therefore the entire attack surface a compromised process has against the kernel. seccomp-BPF (secure computing, filter mode) lets a process attach a small classic-BPF program that the kernel runs on every syscall that process makes. The program looks at the syscall number, the architecture, and optionally the raw argument values, then returns a verdict: allow it, block it with an errno, trap it, or kill. Once installed the filter is irreversible and inherited across any thread the process later spawns — you cannot loosen it, only ever have installed a tighter one.

The reason this is a security primitive and not just a debugging tool is the ordering: the filter runs before the syscall executes. A denied call never reaches the vulnerable kernel code path at all. It is not detect-and-clean-up; it is the door refusing to open. And because a good filter is deny-by-default — anything not explicitly listed loses — a syscall you never anticipated, including one added to the kernel years after you shipped, is denied automatically. An allowlist fails closed. That property is the whole game.

Why filter the VMM and not just the guest

The instinct is that KVM already does the isolation, so what's left to filter? KVM defends the guest-to-host boundary: guest code runs on virtual CPUs and cannot make a syscall into the host at all — its only exit is a hardware VM-exit trap. That boundary is real and it is what limits the blast radius to a single VM. But the Firecracker VMM is itself ordinary host code. It parses whatever the guest writes to its emulated devices, handles the API socket, and manages guest memory. A memory-safety slip in a device model, a logic flaw in a virtio descriptor path, a bad assumption in the API handler — any of these could hand an attacker control of the VMM process without ever formally escaping KVM.

If that process runs unfiltered, the attacker inherits everything the VMM could do: open arbitrary files, execve a shell, open sockets to anywhere, ptrace neighbors. The seccomp filter is the answer to that specific failure. It confines the host-side process so that a full compromise of the VMM lands in something that can make almost no useful syscalls. KVM changes the blast radius; seccomp hardens the thin sliver of host-side surface — the VMM and its device emulation — that sits underneath KVM. You want both because each is designed on the pessimistic assumption that the other already failed.

The filter's actions: Allow, Errno, Trap, KillThread

Firecracker ships an advanced filter, not a flat allowlist. Each syscall in the policy carries a per-syscall action, and the filter can match on argument values, not just the syscall number — so a call like ioctl can be permitted for the exact KVM operations the VMM issues and denied for everything else. The actions that show up in the policy:

• Allow — let the syscall proceed normally. This is the verdict for the small, audited set the VMM legitimately needs: the KVM ioctls, epoll for its event loop, read/write for guest and socket I/O, mmap/munmap for guest memory, futex, and a handful more.
• Errno — block the syscall but let the thread keep running, returning a chosen errno instead. Useful for calls the VMM might probe but shouldn't actually perform, where a clean error is safer than terminating the thread.
• Trap — raise SIGSYS on the offending call so it can be caught and turned into a controlled failure rather than silent success, which is handy for observing and tightening a policy.
• KillThread (the default) — terminate the thread that made a disallowed syscall immediately. Deny-by-default with kill means an unexpected syscall doesn't return an error the attacker can probe around; the thread that tried it simply dies before the kernel does any of the work.

The default verdict for anything not named in the policy is the harsh one on purpose. A blocked call that quietly returns EPERM invites an attacker to try the same goal a different way; a blocked call that kills the thread outright gives them nothing to iterate against. Firecracker splits its filters per thread category too — the vCPU threads, the API thread, and the main VMM thread each have a policy shaped to what that thread actually does — so the vCPU threads (the ones running guest-driven code) carry the tightest allowlist of all.

The JSON filter format Firecracker compiles

You don't hand-write classic BPF for this — that way lies madness. Firecracker keeps its default filters as JSON in the source tree and compiles them into cBPF with a build tool (seccompiler), producing a bytecode blob the VMM installs on itself at startup. Keeping the policy as data rather than hand-rolled bytecode is what makes it auditable: you can read exactly which syscalls the VMM is allowed to make, per thread, per architecture. A single rule looks roughly like this — a syscall name, an action, and optional argument matching:

{
  "default_action": "kill_thread",
  "filter_action": "allow",
  "filter": [
    { "syscall": "read" },
    { "syscall": "write" },
    { "syscall": "epoll_wait" },
    { "syscall": "mmap" },
    { "syscall": "munmap" },
    { "syscall": "futex" },

    {
      "comment": "ioctl is allowed ONLY for specific KVM ops -- arg-matched, not blanket",
      "syscall": "ioctl",
      "args": [
        { "index": 1, "type": "dword", "op": "eq", "val": 44658 }
      ]
    }

    // execve, ptrace, socket, and friends are simply ABSENT.
    // default_action = kill_thread -> attempting one kills the thread.
  ]
}

Verifying the filter is on

This isn't a claim you have to take on faith. When a process has a seccomp filter loaded, the kernel exposes it in the process status file, and you can read it straight off a running Firecracker VMM. A Seccomp mode of 2 means filter mode (mode 1 is the ancient strict mode; 0 is off):

# Find the firecracker VMM process and inspect its seccomp state.
pid=$(pgrep -n firecracker)

grep -E 'Seccomp|Seccomp_filters' /proc/$pid/status
# Seccomp:         2        <- 2 = SECCOMP_MODE_FILTER (a BPF filter is installed)
# Seccomp_filters: 1        <- number of filters attached to the task

# Per-thread: the vCPU threads carry the tightest policy of all.
for t in /proc/$pid/task/*; do
  printf '%s ' "$(basename "$t")"; grep '^Seccomp:' "$t/status"
done

If that reads 0, the VMM is running with no syscall filter — which Firecracker lets you do for debugging but which you should treat as a red flag in anything running untrusted code. A hardened deployment shows mode 2 on every thread.

Two independent walls, on purpose

Put it together and running untrusted code on Firecracker means clearing two independent boundaries in sequence, each enforced by a different mechanism. First the CPU's virtualization extensions confine the guest to its own kernel — to touch the host at all, guest code has to defeat the hypervisor through the narrow virtio interface. Then, if it somehow does, it lands not in a capable host process but in a seccomp-confined one that can call only a few dozen vetted syscalls and dies on the rest. Neither wall is asked to be perfect. The VM boundary is small and heavily audited, but the filter assumes it might fall; the VMM is memory-safe Rust, but the filter assumes it might have a bug anyway. That is what defense in depth means concretely: each layer is built on the assumption the other already failed.

It's also worth being precise about what seccomp does and doesn't buy here. A bare seccomp-confined container is a single layer — a narrowed process on the shared host kernel, and if a must-allow syscall reaches a kernel bug, the whole host and every co-tenant is exposed. Firecracker uses seccomp where it is strongest: not as the boundary that keeps the guest away from the host (KVM is that), but as the backstop hardening KVM's small host-side surface. The filter narrows the door; KVM is what gives each guest its own building.

How PandaStack relies on this

PandaStack runs AI-generated and otherwise untrusted code — code-interpreter sandboxes, managed databases, git-driven hosted apps — and the reason we can hand those workloads a real shell without flinching is this exact layering. Every workload is a KVM-backed Firecracker microVM (its own guest kernel, so the blast radius is one VM), the VMM runs under a privilege-dropping jailer, and seccomp-BPF confines the VMM's own syscalls so a device-emulation breakout hits a straitjacket rather than the host. None of that hardening shows up as latency at create time: because every create restores a baked snapshot on demand rather than cold-booting, sandboxes come live at roughly 179ms p50 (around 203ms p99), with a cold first boot near 3s that happens once per template. The security boundary is rebuilt per VM, not shared across tenants.

The takeaway is the same one Firecracker's own authors bet AWS Lambda on: "it runs in a VM" is necessary but not sufficient for untrusted code. The VM protects the host from the guest; seccomp on the VMM protects the host from the VMM. The PandaStack core is open source under Apache-2.0, so you can read the exact filter the VMM ships with, check Seccomp mode 2 on a live process yourself, and run the whole stack on your own KVM hosts.