Firecracker config file vs REST API: two ways to boot a microVM

There are two genuinely different ways to tell Firecracker what to boot. One: hand it a single static document before it starts, and it configures and starts itself with no further conversation. Two: launch the VMM with nothing decided yet, then drive it step by step — PUT the boot source, PUT the drives, PUT the network, and finally PUT the action that flips the switch. Both end with the same guest kernel running the same workload. The document you get there with is not a cosmetic choice, though — it decides who is allowed to make the last-minute call about which rootfs, which subnet, and how much memory, and when that decision has to be made.

I'm Ajay. I build PandaStack, an open-source Firecracker microVM platform, so I have a stake in this: PandaStack is API-driven end to end, for reasons I'll get into. That's a bias worth naming up front rather than pretending this is a neutral comparison of two equally weighted options for every use case, because it isn't — but the config-file path is real, useful, and the right tool often enough that it deserves a straight explanation rather than a footnote.

Two ways to describe a machine

Every Firecracker VM needs the same handful of facts settled before it can run: which kernel image, which boot arguments, which block devices back which drive IDs, which host tap each network interface rides on, and how many vCPUs and how much memory the guest gets. Firecracker doesn't care how those facts arrive. It cares that they arrive in the right shape, before the transition that freezes them.

The API-driven path treats that transition as a moment you control explicitly: you PUT each resource as its own request, in any order you like, right up until you PUT the action that starts the instance. Nothing is fixed until you say so. The config-file path collapses all of that into one document that's read once, at process launch, and applied in a single pass — by the time you see the process running, the decision window has already closed. Same destination, opposite posture toward when the decision gets locked in.

The API-driven path: decide right before you boot

This is the model most orchestrators use, PandaStack included, and it looks like the walkthrough in our REST API post: start the VMM with an empty socket, then PUT the machine into existence one resource at a time, then PUT the action that starts it. In production this typically runs inside the jailer, which chroots the VMM and drops it to an unprivileged uid/gid before the socket is ever touched — the API surface and the privilege boundary are separate concerns, but they usually ship together.

#!/usr/bin/env bash
set -euo pipefail

# The jailer forks+execs firecracker into a chroot, dropping to an
# unprivileged uid/gid before the VMM ever opens the API socket.
# Exact jailer flags vary by version -- this is the shape, not a
# copy-pasteable invocation for your build.
sudo jailer --id sbx-7f3a --exec-file /usr/bin/firecracker \
  --uid 123 --gid 100 --chroot-base-dir /srv/jail -- \
  --api-sock /run/firecracker.socket

SOCK=/srv/jail/firecracker/sbx-7f3a/root/run/firecracker.socket
until [ -S "$SOCK" ]; do sleep 0.01; done

# Nothing is decided yet. The orchestrator picks the rootfs path, the
# tap name, and the memory size RIGHT NOW -- moments before boot, not
# baked into a file that was written earlier.
curl -s --unix-socket "$SOCK" -X PUT 'http://localhost/boot-source' \
  -H 'Content-Type: application/json' \
  -d '{"kernel_image_path": "/srv/vmlinux-5.10", "boot_args": "console=ttyS0 reboot=k panic=1 pci=off"}'

curl -s --unix-socket "$SOCK" -X PUT 'http://localhost/drives/rootfs' \
  -H 'Content-Type: application/json' \
  -d '{"drive_id": "rootfs", "path_on_host": "/srv/pool/sbx-7f3a/rootfs.ext4", "is_root_device": true, "is_read_only": false}'

curl -s --unix-socket "$SOCK" -X PUT 'http://localhost/network-interfaces/eth0' \
  -H 'Content-Type: application/json' \
  -d '{"iface_id": "eth0", "host_dev_name": "vg-sbx7f3a", "guest_mac": "06:00:AC:C8:03:9A"}'

curl -s --unix-socket "$SOCK" -X PUT 'http://localhost/machine-config' \
  -H 'Content-Type: application/json' \
  -d '{"vcpu_count": 2, "mem_size_mib": 4096, "smt": false}'

# Only now does the machine become real.
curl -s --unix-socket "$SOCK" -X PUT 'http://localhost/actions' \
  -H 'Content-Type: application/json' \
  -d '{"action_type": "InstanceStart"}'

The thing to notice is where the values came from. `rootfs.ext4` under `/srv/pool/sbx-7f3a/` was reflinked into existence a moment earlier by a create request. `vg-sbx7f3a` is a veth already sitting in a pre-allocated network namespace. `mem_size_mib` came from whichever template's baked snapshot this sandbox is about to become. None of those were knowable when the orchestrator's code was written — they're knowable only at the instant a specific request for a specific sandbox arrives, and the API-driven model is what lets that instant be the moment the values get bound.

The config-file path: decide once, boot immediately

Firecracker also accepts a flag — commonly `--config-file`, though verify the exact spelling for your build — pointing at a single JSON document that mirrors the same resources as the API bodies: boot source, machine config, drives, network interfaces, and usually vsock and logger settings too. Firecracker parses the whole thing at process launch, applies every field, and starts the microVM without you making a single API call. There is no `InstanceStart` request to send, because the document itself is the trigger — reading it to completion is what starts the machine.

{
  "boot-source": {
    "kernel_image_path": "/srv/vmlinux-5.10",
    "boot_args": "console=ttyS0 reboot=k panic=1 pci=off"
  },
  "drives": [
    {
      "drive_id": "rootfs",
      "path_on_host": "/srv/rootfs.ext4",
      "is_root_device": true,
      "is_read_only": false
    }
  ],
  "network-interfaces": [
    {
      "iface_id": "eth0",
      "host_dev_name": "tap0",
      "guest_mac": "06:00:AC:10:00:02"
    }
  ],
  "machine-config": {
    "vcpu_count": 2,
    "mem_size_mib": 1024,
    "smt": false
  },
  "vsock": {
    "guest_cid": 3,
    "uds_path": "/srv/v.sock"
  },
  "logger": {
    "log_path": "/srv/logs/fc.log",
    "level": "Info"
  }
}

Treat that as an illustrative shape, not a byte-exact schema to copy into a production file — the actual required and optional fields, and whether every one of those top-level keys is still spelled that way, depends on the Firecracker version you're running. What's stable across versions is the idea: one file, read once, applied in order, machine running by the time the process would otherwise have printed its next log line. Whether the API socket stays up afterward for post-boot operations like pausing or snapshotting, or whether a separate flag suppresses the control surface entirely for a truly hands-off boot, is again the kind of detail to confirm against your version's docs rather than assume.

This is a genuinely good fit for a specific shape of problem: a fixed, single-purpose VM whose kernel, rootfs, and network setup are known ahead of time and don't need to change per boot. Kernel bring-up while you're iterating on boot args. A CI job that always boots the same test image against the same tap device. A minimal supervisor script — a few lines of shell, not a control plane — that just needs one VM up with no orchestration logic wrapped around it. In all of those, there's no runtime decision being made; the config file just is the decision, written down once.

API-driven vs config-file, compared

• Setup model — API: a sequence of PUT requests over a socket, applied incrementally, ending in an explicit InstanceStart action. Config file: one JSON document read once at process launch, applied in a single pass with no separate start call.
• Who decides the values — API: whatever process is holding the socket, at the moment it sends each request — a scheduler, a create handler, anything with runtime state. Config file: whoever wrote the file, at the moment it was written — which has to be before the Firecracker process even starts.
• When the decision has to be made — API: as late as you want, including microseconds before InstanceStart. Config file: baked in ahead of time; there's no hook to substitute a different rootfs path once the process has read the file.
• Fit for orchestrated fleets — API: this is the only workable model once you have more than a handful of VMs, because each one needs its own rootfs clone, its own tap, its own memory size pulled from a template that might change between deploys. Config file: awkward here — you'd be generating a fresh file per VM anyway, which is most of the complexity of the API path with none of its flexibility.
• Fit for a single fixed VM — API: works, but is more ceremony than the job needs — a handful of curl calls or SDK calls just to boot one thing that never changes. Config file: exactly the right amount of mechanism — one file, one flag, done.
• Relationship to snapshot and restore — API: snapshot creation and restore are themselves API operations (pause, then PUT to the snapshot-create endpoint; PUT to the snapshot-load endpoint, then resume), so any workflow built around snapshots is API-driven by necessity, regardless of how the very first cold boot happened. Config file: can produce the VM that gets snapshotted, but has no equivalent for restoring one — there's no static document that says 'become this already-running machine.'
• Debuggability — API: each call fails independently with its own status code and fault message, so you know exactly which resource was wrong. Config file: a bad field fails the whole boot at once, before you have a socket to introspect with — you're debugging from logs and exit codes, not a live API you can poke.
• Observability into intent — API: the sequence of calls itself is a record of what was decided and when, which is useful if you're logging every request an orchestrator makes. Config file: the file itself is the record — clear for a fixed VM, but it tells you nothing about a decision process because there wasn't one to record.
• Best fit — API: fleets, orchestrators, anything doing snapshot-restore, anything where the exact machine shape is a runtime decision. Config file: fixed single-purpose VMs, kernel and rootfs bring-up, CI, minimal supervisor scripts that don't want to run a control loop just to boot one thing.

Why a snapshot-restore fleet has to be API-driven

PandaStack's create path makes the choice for us, structurally. Every sandbox create restores a template's baked snapshot rather than cold-booting — that's how the platform gets a p50 of 179ms (p99 around 203ms) instead of the roughly 3 seconds a cold boot takes. Restoring a snapshot is `PUT /snapshot/load` followed by a resume action. There is no config-file equivalent of 'load this snapshot' — the config file describes a machine to construct from a kernel and a rootfs, not a frozen machine to reconstitute from a memory image and a state file. The moment your platform's hot path is snapshot-restore, you're on the API by definition, not by preference.

Layered on top of that, every single create needs values that don't exist until the request does. The rootfs is a fresh reflinked clone of the template's disk, created for this sandbox alone. The network identity comes out of a pool of 16,384 pre-allocated /30 subnets per agent — NATID slots that get bound to this specific sandbox at allocation time, not decided in advance. The memory size comes from whichever template's `meta.json` this create is restoring, which can differ between templates and can change when a template gets rebaked. None of those three facts are knowable when a hypothetical static config file would have to be written; they're only knowable at the moment a specific create request lands on a specific agent. A config file wants its answers before the process starts. An orchestrator managing a fleet doesn't have its answers until then.

It's the same reason fork inherits the API-driven model rather than needing its own mechanism: forking a running sandbox is conceptually a snapshot-restore pointed at a live VM's state instead of a template's, and it's fast for the same reason — 400–750ms same-host, 1.2–3.5s cross-host where the memory and rootfs artifacts have to travel first. Managed database creates go through the same restore path underneath their own bootstrap, which is why they land in the 30–90 second range rather than the minutes a from-scratch Postgres install would take. All of it is late-bound, all of it is decided by a control plane at request time, and none of it fits inside a document that has to be complete before the VMM process opens its first file handle.

Neither one is the beginner mode

It's tempting to read the config file as the simple option and the API as the advanced one, but that's not quite the right axis. The real question is where the decision about the machine's shape gets made: once, by a person or a script, before anything runs — or repeatedly, by a program, at the exact moment each machine needs to exist. A single test VM you boot the same way every time genuinely doesn't need a control loop wrapped around it, and reaching for the full API there is just extra moving parts for no benefit. A fleet where every create needs its own rootfs, its own subnet, and possibly its own template can't be expressed as a file at all — the whole point is that the values aren't fixed.

If you're standing up your first Firecracker VM to see it boot, the config file is probably less code and fewer moving parts to get a kernel printing to your console. If you're building anything that creates VMs on demand, restores snapshots, or forks running sandboxes, you're going to end up talking to the API regardless of how the very first template got baked — because that's the only door snapshot-restore walks through.