Firecracker vs systemd-nspawn: isolation for untrusted code

systemd-nspawn is the container manager that came free with your init system, and it's genuinely delightful. One command boots a full Linux OS tree — its own PID 1, its own journald, its own network — right there on your host. It's the engine behind mkosi image builds and systemd's portable services, and with machinectl it can register and manage those OS trees like first-class machines. It feels like a virtual machine. So the obvious question when you're eyeing it to run other people's code is the same one we ask of every container: is that VM feeling actually a VM boundary, or a very convincing chroot with better paperwork? The short answer — it's the chroot, and nspawn's own documentation has historically said so out loud.

What systemd-nspawn actually is

Think of systemd-nspawn as chroot's ambitious successor. A plain chroot pins a process to a subdirectory as its root filesystem and calls it a day. nspawn takes that directory (or a disk image, or a btrfs subvolume) and wraps the processes inside it in the full set of Linux container primitives: namespaces for PID, mount, network, UTS, and IPC; cgroups for resource accounting; capability dropping and optional seccomp and user-namespace mapping. The result boots a real init system and behaves like a machine you can `machinectl login` into. mkosi uses it to assemble and test OS images; systemd's portable-services feature uses the same machinery to run a bundled service in its own root tree.

But strip away the ergonomics and the mechanism is identical to every other Linux container. There is exactly one kernel on the box, and every nspawn container runs on it. When a process inside the container makes a syscall, that syscall is serviced by the same host kernel serving every other container and the host itself. nspawn doesn't boot a kernel — it boots a userspace against a kernel that was already running. That's precisely why it's so cheap and so fast: there's no second OS to start, only namespaces drawn around processes that are, underneath, ordinary host processes.

The man page warned you

This isn't an outside critique — it's what the tool told you about itself. For much of nspawn's life, its own manual carried an explicit warning that it was not suitable for secure isolation of untrusted code: the note read, in spirit, that nspawn is not fully secure and that a determined attacker inside the container could break out and gain access to the host. The maintainers have since hardened defaults (user namespaces, stricter device and syscall policy) and softened that language, so you should verify the exact current wording against the upstream systemd docs rather than quote a snapshot. But the underlying reason for the original warning hasn't changed: a container secured with namespaces and seccomp is a policy layer over a shared kernel, and the shared kernel is still the thing executing the untrusted code's syscalls.

nspawn will happily boot an entire OS for you. It will just as happily let that OS negotiate directly with the one kernel everyone on the box is sharing — and a good chunk of the Linux CVE archive is stories about that negotiation going wrong.

User-namespaced nspawn containers raise the bar the same way unprivileged LXD containers do — root inside maps to a harmless high UID outside, seccomp trims the syscall menu, capabilities are dropped. That's real, and it's the right default. It is also not a hardware boundary. User namespaces have themselves been a recurring source of privilege-escalation CVEs, because they hand unprivileged users reachable kernel code paths that used to be root-only. A bug in a syscall your filter didn't block, reached through a code path your policy didn't anticipate, doesn't care what UID the container thinks it is.

How Firecracker moves the boundary

A Firecracker microVM is a different category of thing entirely. It's a hardware-virtualized guest running under KVM, booting its own guest kernel from its own kernel image. Untrusted code inside makes syscalls to the guest kernel — a kernel that is not your host's kernel and shares no state with it. The guest reaches the outside world only through a deliberately tiny set of emulated virtio devices (a block device, a network device, vsock) exposed by the Firecracker VMM, which itself runs as an unprivileged host process behind a seccomp filter and, in production, the jailer.

So to escape, untrusted code must first find a bug in the guest kernel — which only wins it root inside a disposable VM, i.e. nowhere — and then find and exploit a bug in Firecracker's minimal device-emulation surface or the KVM/hardware boundary beneath it. That is a vastly smaller and more heavily audited surface than the full Linux syscall interface an nspawn container is exposed to. It's the same reasoning behind AWS Lambda and Fargate running every customer's code in Firecracker microVMs rather than shared-kernel containers: at multi-tenant scale, one shared kernel is not a boundary you'll bet strangers' code against.

Side by side

• Isolation model — systemd-nspawn: namespaces + cgroups + seccomp/capabilities on the shared host kernel. Firecracker: hardware-virtualized microVM with its own guest kernel under KVM.
• Kernel — systemd-nspawn: the host kernel, shared by every container; you cannot run a different one. Firecracker: a separate guest kernel per VM, and you pick the version.
• Untrusted / AI-generated / multi-tenant code — systemd-nspawn: risky even with user namespaces — a kernel bug escapes the tree. Firecracker: designed for exactly this (the Lambda model).
• What it's built for — systemd-nspawn: building and testing OS images (mkosi), portable services, running a full trusted Linux userspace, even booting machines via machinectl. Firecracker: running untrusted or per-tenant workloads at density.
• What it feels like — systemd-nspawn: a lightweight VM with full systemd inside. Firecracker: a small, fast VM — because it is one.
• Boot / create — systemd-nspawn: very fast, no kernel to boot, just namespaces. Firecracker: cold boot in the low seconds, but snapshot-restore makes per-request creates practical (~49ms restore, p50 ~179ms create).
• Density / overhead — systemd-nspawn: near-zero beyond the container's own processes. Firecracker: a few MB per guest for the kernel + device model.
• Best fit — systemd-nspawn: trusted system containers, OS-image workflows, systemd-native hosting you control. Firecracker: multi-tenant, untrusted, per-user, or model-generated code.

When systemd-nspawn is genuinely the right tool

None of this makes nspawn bad — it makes it a systems tool, not an untrusted-code sandbox. If you control everything running inside the container, the shared-kernel risk is your own, and nspawn's density, speed, and deep systemd integration are excellent. Reach for it happily when:

• You're building or testing OS images with mkosi — spinning up a full root tree to run an install, boot it, and check it works.
• You're running systemd portable services or splitting a host into a few trusted, systemd-native service trees you operate yourself.
• You want a full-OS box per project or per branch that developers can `machinectl login` to and treat like a real machine.
• First-party CI where the code being built is yours and the threat model is accidents, not adversaries.
• You specifically need PID 1 to be real systemd — timers, journald, socket activation — which an application container makes awkward.

When you must reach for a microVM

The dividing line is trust, not workload size. The moment the code inside is untrusted, multi-tenant, or generated at runtime, a shared kernel stops being an acceptable boundary and you want a real VM boundary: AI agents executing model-written shell commands you can't review first, multi-tenant platforms landing different customers' code on shared hosts, per-user code playgrounds and online judges, or untrusted third-party plugins and build steps from repos you don't control.

Here's the same 'boot a machine and run something in it' operation in both worlds. First nspawn, booting a full OS tree the systemd way — ideal for a workload you control:

# systemd-nspawn: boot a full OS tree — great for a workload YOU control.
# Pull a minimal image, then boot it as a registered machine.
sudo machinectl pull-tar https://example.com/ubuntu-24.04.tar.xz noble
sudo systemd-nspawn --boot --machine=noble
#   -> real PID 1 (systemd), journald, its own network namespace.

# Or manage it as a first-class machine:
sudo machinectl start noble
machinectl login noble        # feels exactly like SSH-ing into a VM
uname -r                      # ...but this prints the HOST kernel.
# Shared kernel underneath: a kernel bug here can reach the host.

And the same shape for untrusted code, where each run gets its own guest kernel behind a hardware boundary. This is exactly the line PandaStack is built on — every sandbox is its own Firecracker microVM, and because a create restores a baked snapshot on demand instead of cold-booting, the VM-grade isolation costs you almost nothing in latency (create is p50 ~179ms, p99 ~203ms):

from pandastack import Sandbox

# Firecracker microVM: separate guest kernel, hardware-isolated.
# The right shape for untrusted / model-generated code.
with Sandbox.create(template="code-interpreter", ttl_seconds=300) as sbx:
    result = sbx.exec(
        "python3 -c 'import os; print(os.uname().release)'",
        timeout_seconds=30,
    )
    print("guest kernel:", result.stdout.strip())  # NOT your host's kernel
    print("exit:", result.exit_code)
# VM is destroyed here — blast radius of arbitrary code is one throwaway guest.

Same ergonomics, different boundary. With nspawn the blast radius of a kernel exploit is your host and every neighbor sharing it; with a per-run microVM it's a disposable guest with its own kernel, memory, filesystem, and network namespace. The historical reason people stretched containers over jobs they shouldn't was that VM isolation meant paying multi-second boot times per request. Snapshot-restore removes that excuse: a fresh microVM per untrusted run is now fast enough to do on every request, and forks add another cheap dimension — same-host in 400–750ms, cross-host in 1.2–3.5s.

The honest verdict

systemd-nspawn and Firecracker answer different questions. If your question is 'how do I boot and manage a full Linux OS tree that I trust, with real systemd inside,' nspawn is a superb answer and a microVM would be overkill — it's why mkosi and portable services lean on it. If your question is 'can this safely run code I don't control,' a shared-kernel container, user-namespaced or not, says 'probably, until it doesn't' — and its own manual historically said as much — while a hardware-isolated microVM says 'yes.' Pick the boundary that matches your trust model, and verify the current hardening defaults against each project's docs, since these move over time.