Firecracker's Rate Limiter, Explained

Put a few hundred untrusted sandboxes on one host and the failure mode is not usually an escape — it's a hog. One tenant kicks off a `dd` loop against its disk, or opens a firehose download, and suddenly every other guest on the box is doing I/O through a straw. Nothing crashed, nothing was breached; one neighbor just ate the whole floor's bandwidth. Firecracker's answer to that is a per-device rate limiter: a token bucket you attach to a guest's virtio-block drive or virtio-net interface that caps how much I/O it can push, so a loud tenant can't saturate disk or network for the quiet ones. This post explains what the token bucket actually is, the three fields that describe it (`size`, `refill_time`, `one_time_burst`), how bandwidth and ops buckets differ, how you wire a limiter onto a drive and a NIC through Firecracker's API, and where the limiter ends and cgroups plus the jailer begin. The Firecracker API details here are the ones you configure by hand; where a field name is worth double-checking, I'll say so — pin it against the Firecracker API spec for your version before you ship.

The token bucket, concretely

A token bucket is a bouncer that only lets so many bytes into the club per second. The bucket holds tokens. Every unit of work — a byte transferred, or an I/O operation performed — costs a token. When work arrives, it spends tokens from the bucket; if the bucket has enough, the work proceeds immediately. If the bucket is empty, the work waits until tokens refill. Meanwhile the bucket refills at a steady rate. That's the whole model: a reservoir that drains as fast as the guest does work and refills at a fixed pace, and when it hits empty the guest is throttled to the refill rate.

The elegance is that this gives you a smooth average rate and a controlled burst in the same two numbers. The refill rate is the sustained ceiling — over any long window, the guest can't average more than the bucket refills. But because the bucket has capacity, a guest that was idle for a moment accumulates tokens and can spend them in a rush, absorbing a spike without being clamped to a rigid per-tick limit. You get "no more than X per second on average, but a short burst up to the bucket's capacity is fine." That's exactly the shape of real traffic — mostly quiet with occasional spikes — which is why token buckets, not fixed-window counters, are what Firecracker uses.

The three fields: size, refill_time, one_time_burst

Firecracker describes a bucket with a small object, and the two fields that always matter are `size` and `refill_time`. Together they define the rate. A third field, `one_time_burst`, adds a one-shot allowance for the startup spike.

• size — the bucket capacity: the maximum number of tokens it can hold. For a bandwidth bucket, tokens are bytes, so size is a byte count. For an ops bucket, tokens are operations, so size is a count of I/O operations. This is also the largest burst the guest can spend at once when the bucket is full.
• refill_time — how long, in milliseconds, it takes to refill the bucket from empty back to size. This is what sets the sustained rate: rate = size / refill_time. A size of 100 MiB with a refill_time of 1000 ms is ~100 MiB/s sustained; the same size with a refill_time of 100 ms is ~10× faster. Two knobs, one rate — pick the size for the burst you'll tolerate and the refill_time for the average you'll allow.
• one_time_burst — an optional extra pool of tokens, spent once, on top of the normal bucket. It does not refill. It exists for the workload that legitimately needs a big spike at the very start — reading a rootfs on boot, an initial bulk fetch — and then settles into a steady rate. The guest gets its one big gulp from the burst pool, and once it's drained, it's governed purely by size and refill_time thereafter. (Confirm the exact field name against the Firecracker API spec for your version — the burst allowance is a documented part of the token-bucket config, but spellings drift across releases.)

The mental math to internalize: `refill_time` is the denominator of your rate. Engineers reach for `size` first because it's the intuitive "how big" number, but on its own it only tells you the maximum instantaneous burst. The sustained throughput a tenant actually gets is `size ÷ refill_time`. If you want 50 MiB/s, there are infinitely many (size, refill_time) pairs that give it — a bigger size with a proportionally longer refill_time buys a larger allowed burst at the same average. That trade — burst tolerance vs. rate — is the real design decision.

Two buckets: bandwidth vs. ops

A single limiter can carry two independent buckets, and they guard against two different abuses. The `bandwidth` bucket meters bytes: it caps how much data a guest can move per second. The `ops` bucket meters operations: it caps how many I/O requests a guest can issue per second, regardless of how big each one is. You can set either, both, or neither, and they're enforced together — a request must have tokens available in every configured bucket to proceed.

Why both? Because bytes and operations are separate resources, and a limit on one doesn't constrain the other. A bandwidth-only limit stops a guest from moving a lot of data, but it does nothing against a flood of tiny operations — thousands of 512-byte reads that barely register on a byte counter but hammer the disk queue and the host's I/O scheduler. That small-random-I/O pattern is exactly what wrecks a shared spindle or an overworked SSD, and only the ops bucket catches it. Conversely, an ops-only limit lets a guest move enormous data in a few huge requests. For untrusted multi-tenant workloads you usually want both: bandwidth to bound throughput, ops to bound request rate.

• Bandwidth bucket — tokens are bytes. Caps data throughput (e.g. bytes/second on a disk or NIC). Stops the firehose: bulk downloads, big sequential disk writes. Blind to a swarm of tiny requests.
• Ops bucket — tokens are operations. Caps request rate (IOPS on a drive, packets-per-second-ish work on a NIC). Stops the storm of tiny I/Os that starve the queue without moving much data. Blind to a few enormous transfers.
• Use both — bandwidth bounds how much data moves; ops bounds how many operations are issued. Together they cover the two orthogonal ways one guest degrades a host: too many bytes and too many requests. Setting only one leaves the other axis wide open.

Configuring a limiter on a drive

Like every Firecracker device, the rate limiter is wired up through the REST-over-Unix-socket API. On a block device it lives in the same drive object you PUT before boot — a `rate_limiter` field alongside the path and engine. Here's a rootfs drive limited to roughly 100 MiB/s of bandwidth and 2,000 operations per second, with a one-time burst so the boot-time rootfs read isn't throttled by a cold bucket.

// PUT /drives/rootfs — a virtio-block device with a token-bucket limiter.
// bandwidth: ~100 MiB/s sustained (size bytes / refill_time ms).
// ops:       ~2000 IOPS sustained.
// one_time_burst lets the boot-time rootfs read spend a big one-shot
// allowance before the steady-state limit clamps down.
{
  "drive_id": "rootfs",
  "path_on_host": "/var/lib/pandastack/vms/demo/rootfs.ext4",
  "is_root_device": true,
  "is_read_only": false,
  "io_engine": "Async",
  "rate_limiter": {
    "bandwidth": {
      "size": 104857600,
      "one_time_burst": 209715200,
      "refill_time": 1000
    },
    "ops": {
      "size": 2000,
      "refill_time": 1000
    }
  }
}

Read the numbers: the bandwidth bucket holds 104,857,600 tokens (100 MiB), refilling fully every 1,000 ms — so ~100 MiB/s sustained, with a one-time pool of 209,715,200 tokens (200 MiB) the guest can spend once at the start. The ops bucket holds 2,000 tokens refilling every 1,000 ms — ~2,000 operations per second. The limiter sits in front of the drive's FileEngine, so it governs how much I/O is permitted whether the drive runs the Sync or the async io_uring engine underneath. The engine decides how efficiently permitted I/O is serviced; the limiter decides how much is permitted.

Configuring a limiter on a network interface

The network interface takes the same token-bucket shape, with one extra wrinkle: a NIC has two directions, and Firecracker lets you limit them separately. The network-interface config exposes an `rx_rate_limiter` (traffic into the guest — receive) and a `tx_rate_limiter` (traffic out of the guest — transmit), each a full token-bucket limiter with its own bandwidth and ops buckets. That separation matters because inbound and outbound abuse look different: a scraper fleet cares about egress (transmit), a download-heavy guest about ingress (receive), and you often want asymmetric caps.

// PUT /network-interfaces/eth0 — a virtio-net device with separate
// receive (rx) and transmit (tx) token-bucket limiters.
// Here: symmetric ~50 MiB/s each way, ops left unlimited.
{
  "iface_id": "eth0",
  "host_dev_name": "tap0",
  "guest_mac": "06:00:AC:10:00:02",
  "rx_rate_limiter": {
    "bandwidth": { "size": 52428800, "refill_time": 1000 }
  },
  "tx_rate_limiter": {
    "bandwidth": { "size": 52428800, "refill_time": 1000 }
  }
}

Same math, two directions: each 52,428,800-token (50 MiB) bandwidth bucket refilling every 1,000 ms gives ~50 MiB/s in that direction. Leave `ops` off and only bandwidth is metered; add an `ops` bucket to each direction if you also want to bound packet-rate-ish work. If you only care about capping how hard a guest can hit the outside world — the common case for untrusted egress — you can set just `tx_rate_limiter` and leave `rx_rate_limiter` unset. (Field names `rx_rate_limiter` / `tx_rate_limiter` are the documented network-interface knobs, but as always, verify against the Firecracker API spec for your release.)

The limiter vs. cgroups vs. the jailer

The rate limiter is not the only resource lever around a Firecracker VM, and it's important to know where it stops. Firecracker runs each microVM as a process, and the recommended way to run it in production is under the jailer — a wrapper that drops the VMM into a restricted, cgroup-scoped, namespaced sandbox before it ever touches KVM. Those two mechanisms — the in-VMM token-bucket limiter and the host's cgroups (applied by the jailer or your orchestration) — solve overlapping-but-distinct problems, and you generally want both.

• Rate limiter (in Firecracker) — per-device, per-guest I/O shaping. Meters bytes and operations on a specific virtio-block drive or virtio-net NIC via token buckets. It's fine-grained (this drive, that direction) and lives inside the VMM. It does not limit CPU, host memory, or the number of file descriptors — it only shapes I/O on the devices you attach it to.
• cgroups (via the jailer) — host-kernel resource control on the Firecracker process as a whole. Caps CPU shares, pins the VMM to specific cores, limits memory and PIDs, and can cap block-device I/O at the cgroup level too. Enforced by the host kernel regardless of what the guest does — it needs no cooperation and covers resources the rate limiter never touches.
• The jailer (isolation, not shaping) — not a rate limiter at all. It's the security wrapper: chroot, dropped capabilities, a dedicated network/mount namespace, seccomp, and the cgroup setup above. It's what makes a compromised VMM process a much smaller blast radius. Rate limiting rides inside that jail; the jail is the containment.
• Rate limiter vs. cgroups — the limiter is device-scoped and lives in the VMM (great for asymmetric per-NIC/per-drive shaping); cgroups are process-scoped and live in the host kernel (great for hard CPU/memory ceilings the guest cannot evade). Use the limiter to shape a guest's disk and network I/O; use cgroups to bound its CPU and memory. Neither replaces the other.

Why this matters for multi-tenant density

PandaStack's economics rest on packing many microVMs onto one KVM host — the networking layer alone pre-allocates 16,384 /30 subnets per agent so per-sandbox network isolation is never the bottleneck. But isolation of the security kind (KVM plus the minimal virtio device model) says nothing about fairness. Two tenants can be perfectly isolated from each other's data and still fight over the same disk queue and the same NIC. The token-bucket limiter is the fairness layer: it lets the platform hand each sandbox a bounded slice of I/O so one tenant's `dd` loop or bulk download doesn't turn every neighbor's create and exec into a slow crawl. That fairness is what makes dense multi-tenancy actually usable rather than just technically secure.

It also protects the parts of the platform that are latency-sensitive by design. A create lands at a 179ms p50 (p99 ~203ms) by restoring a baked snapshot rather than the ~3s of a first cold boot, and that restore reads rootfs blocks off the host file — a budget tight enough that a neighbor saturating the disk is not an abstract worry. Same story for forks: a same-host fork runs in roughly 400–750ms sharing the parent's blocks copy-on-write, and a cross-host fork is 1.2–3.5s once artifacts move over the network — both paths care that no single guest is monopolizing disk or network. And a managed Postgres microVM, which takes 30–90s to create and then serves a steady stream of small writes, is precisely the workload you want to protect with an ops bucket so a noisy neighbor can't starve its I/O. PandaStack's core is open source under Apache-2.0, so you can run the per-host agent on your own KVM hosts, attach a rate_limiter to a drive or NIC, and watch a noisy guest get clamped while its neighbors keep their throughput. For the engine that services permitted disk I/O, see /blog/firecracker-io-uring-block-io; for the broader device model the limiter attaches to, /blog/firecracker-virtio-devices; and for the isolation wrapper it rides inside, /blog/firecracker-jailer-explained.

From the SDK, you don't touch any of this by hand — the platform attaches sensible per-device limiters when it creates a sandbox, and you just get a fair slice. But the primitive underneath is exactly the token bucket above, and it's worth knowing it's there the day a tenant's workload legitimately needs a higher ceiling and you reach for a custom template with roomier buckets.

from pandastack import PandaStack

ps = PandaStack()  # reads PANDASTACK_API_KEY

# Each sandbox comes up behind per-device token-bucket limiters, so a
# neighbor hammering its disk can't steal this one's I/O. You just run.
sbx = ps.sandboxes.create(template="code-interpreter")
out = sbx.exec("dd if=/dev/zero of=/tmp/blob bs=1M count=512")
print(out.stdout)  # throttled to the drive's bandwidth bucket, not the raw disk
sbx.delete()