Why Firecracker Uses virtio-MMIO, Not virtio-PCI

If you've configured a QEMU guest, you know the ritual of PCI: devices sit on a virtual PCI bus, the guest enumerates them by walking PCI config space, and each gets a BAR (base address register), an interrupt line, maybe MSI-X vectors. It's the same discovery machinery a physical PC uses, faithfully emulated. Firecracker does none of this. There is no PCI bus in a Firecracker microVM. Its virtio devices are wired over MMIO — memory-mapped I/O at fixed physical addresses — and the guest is simply told where they are. This post is about that choice: what virtio-MMIO actually is, how it differs from virtio-PCI, and why dropping an entire bus is one of Firecracker's better security decisions.

I'm Ajay; I built PandaStack on Firecracker, and the minimal device model is a big part of why the security story holds and why boot and restore stay fast. A device you didn't emulate can't have a bug; a bus you don't have can't be fuzzed.

The two virtio transports

virtio itself — the paravirtualized device standard where guest and host cooperate through shared-memory virtqueues instead of emulated hardware registers — is transport-agnostic. The device logic (a virtio-block, a virtio-net) is the same regardless of how the guest finds the device and rings its doorbell. That "how the guest finds it" is the transport, and there are two that matter.

virtio-PCI

In the PCI transport, each virtio device presents as a PCI device on a virtual bus. The guest discovers it by scanning PCI config space, reads its capabilities to locate the virtio structures, maps its BARs, and sets up interrupts — usually MSI-X, which gives each virtqueue its own interrupt vector. This is rich and standard: it supports arbitrary device counts, hotplug (add a device to a running guest), and fine-grained per-queue interrupts. The cost of that richness is a lot of emulated surface: a PCI host bridge, config space, capability parsing, MSI-X tables — all code the VMM must implement and all input the guest can poke at.

virtio-MMIO

In the MMIO transport, there is no bus and no discovery protocol. Each device is a block of registers at a fixed guest-physical address. The guest doesn't scan for it; it's told the address, the size, and the interrupt line out of band — on x86 Firecracker declares them on the kernel command line as `virtio_mmio.device=` entries; on other platforms a device tree does the same job. The guest driver reads a magic value and version at the device's base address to confirm what it is, then talks to it through that register window. No config space, no BARs, no MSI-X. It's the bare minimum needed to locate a virtqueue and kick it.

# How the guest LEARNS about MMIO devices: it's told, on the kernel cmdline.
# (Illustrative -- Firecracker builds these boot args for you.)
#   virtio_mmio.device=<size>@<base-addr>:<irq>
boot_args="console=ttyS0 reboot=k panic=1 pci=off \
  virtio_mmio.device=4K@0xd0000000:5 \   # a virtio device at a fixed address
  virtio_mmio.device=4K@0xd0001000:6"    # another, next window over

# Note 'pci=off': the guest kernel isn't even asked to probe a PCI bus,
# because there isn't one. Fewer subsystems initialized = faster, smaller boot.

That `pci=off` on the boot line is not incidental. Firecracker's guests skip PCI probing entirely, which trims kernel init work and removes a whole subsystem from the boot path. The device model the VMM presents is correspondingly tiny: a handful of MMIO device windows, a serial port, and not much else.

Why Firecracker chose MMIO

Firecracker's design brief was a VMM for running untrusted, multi-tenant workloads (Lambda, Fargate) with the smallest plausible trusted computing base. Every emulated device is attack surface: it's code that parses guest-controlled input from inside the security boundary. A PCI bus is a lot of such code — config space, capability lists, BAR programming, MSI-X — and most workloads need exactly one disk, one network interface, and a vsock. So Firecracker made the ruthless call: don't emulate the bus, don't emulate any device you don't strictly need, and wire the few you do keep over the simplest transport available.

• Attack surface — virtio-PCI: a PCI host bridge, config space, BAR programming, and MSI-X are all emulated code taking guest input. virtio-MMIO: a fixed register window per device and nothing else — dramatically less code for a guest to probe or exploit.
• VMM size — virtio-PCI: the device model is larger, which enlarges the trusted computing base. virtio-MMIO: a minimal device model keeps the VMM small and auditable, part of Firecracker's tiny-TCB goal.
• Boot cost — virtio-PCI: the guest probes the PCI bus at boot. virtio-MMIO: `pci=off`; the guest skips PCI init entirely, one less subsystem on the boot path.
• Flexibility — virtio-PCI: arbitrary device counts, hotplug, per-queue MSI-X interrupts. virtio-MMIO: fixed set of devices declared at boot, no hotplug — a deliberate limit, not an oversight.
• Who uses which — virtio-PCI: QEMU and general-purpose VMs that need the richness. virtio-MMIO: Firecracker and minimal microVMs that value a small, fixed device model over flexibility.

The most secure device is the one you never emulated. Firecracker's device model is small on purpose, and MMIO is the transport that lets it stay that way.

The tradeoffs you accept

This isn't free, and Firecracker doesn't pretend otherwise. Dropping PCI means dropping the things PCI is good at. There's no hotplug — you can't add a disk or NIC to a running Firecracker guest the way you might with a PCI-based VM; the device set is fixed at boot. The device count is bounded by the MMIO windows declared on the cmdline. And you don't get MSI-X's per-queue interrupt richness. For a general-purpose hypervisor those would be real limitations. For a microVM whose job is to boot fast, do one bounded task, and be thrown away, they're exactly the features you're happy to trade for a smaller, safer VMM.

For a sandbox platform, the MMIO choice compounds nicely. A small device model means a small VMM, which means less to harden and a smaller thing to reason about when you're running strangers' code. It also means fast boots and fast restores, because there's simply less machinery to bring up. On PandaStack every create restores a baked snapshot on demand (restore step ~49ms, p50 179ms), and part of why that's cheap is that the guest coming back to life has a minimal device model to reinitialize — a few MMIO windows, not a PCI topology. The security argument and the performance argument turn out to be the same argument: keep the VMM small.

For the devices themselves, /blog/firecracker-virtio-devices walks the virtio model in general; the security rationale for a minimal VMM is in /blog/firecracker-vmm-security-model; and why the whole thing boots so fast is in /blog/how-firecracker-boots-fast.