UFFDIO_ZEROPAGE vs UFFDIO_COPY: Stop Paying RAM for Zeros

Here is a fact that took me embarrassingly long to internalise: when you snapshot a running microVM with 4 GiB of RAM, the resulting memory file is 4 GiB, and almost none of it is interesting. A guest that has booted a kernel, started an init, opened a listening socket and gone quiet is not using 4 GiB of anything. It is using a modest working set surrounded by an ocean of never-touched heap arenas, unpopulated page cache, and free-list pages that were zeroed once and never looked at again. The hypervisor doesn't know that. To the hypervisor it's a flat file, and every byte of it has to be answerable the instant a vCPU asks.

I'm Ajay; I build PandaStack, a Firecracker microVM platform where every sandbox create is a snapshot restore rather than a boot. That means we run a userfaultfd handler in production, on the critical path of every vCPU in the fleet, and we've had to care rather a lot about one line of it: the branch that decides how a fault gets answered. This post is about that branch — what `UFFDIO_COPY` and `UFFDIO_ZEROPAGE` actually do to the kernel's page tables, and why the difference is measured in host density rather than milliseconds. I'll also be honest about when none of this matters to you.

Most of a restored guest's memory is nothing

A demand-paged restore works like this: instead of reading the whole memory file into the guest's address space before resuming, you resume immediately with an empty address space and fill pages in as the guest touches them. The guest boots to a prompt having faulted a small fraction of its configured RAM. That's the entire point — it's why a restore can be fast and why the memory file can live somewhere other than the local disk.

The naive handler answers every one of those faults identically: work out which offset of the snapshot file the faulting address corresponds to, get 4096 bytes from there, and hand them to the kernel with `UFFDIO_COPY`. It's correct. It's also, for the large majority of faults, a request for the kernel to allocate a brand-new anonymous page and `memcpy` zeros into it. You have built a very sophisticated mechanism for allocating physical memory in order to store nothing, one page at a time, on the critical path of a vCPU.

That cost doesn't show up in your boot-latency dashboard. It shows up two months later when a host that should be running a comfortable number of idle sandboxes starts reclaiming, and you go looking for a leak that doesn't exist.

userfaultfd, in one page

`userfaultfd(2)` hands you the page-fault handling for a region of memory. You get a file descriptor, register a virtual address range against it, and when a thread touches an address in that range whose page-table entry is absent, the kernel parks that thread and posts you a message. You read it, decide what should be there, and answer with an ioctl — which both installs the page and wakes the blocked thread.

The mode that matters here is `UFFDIO_REGISTER_MODE_MISSING`: fire on absent PTEs. (Write-protect mode, which fires on writes to present-but-protected pages, is for dirty tracking and incremental snapshots — a different post.) Firecracker does not create the fd for you: your handler listens on a Unix socket, Firecracker connects, and passes the userfaultfd over it with `SCM_RIGHTS` alongside a description of each guest memory region — host base address, length, page size, and the region's offset inside the snapshot memory file. From then on you own every fault in those regions.

/* Setup + the event loop. Firecracker has already handed us `uffd` over a
 * Unix socket (SCM_RIGHTS), along with a description of each guest memory
 * region: host base address, length, page size, and the region's offset
 * inside the snapshot memory file. */

struct uffdio_api api = { .api = UFFD_API, .features = 0 };
if (ioctl(uffd, UFFDIO_API, &api) < 0)
        die("UFFDIO_API");

struct uffdio_register reg = {
        .range = { .start = (unsigned long)r->base, .len = r->len },
        .mode  = UFFDIO_REGISTER_MODE_MISSING,   /* fire on absent PTEs */
};
if (ioctl(uffd, UFFDIO_REGISTER, &reg) < 0)
        die("UFFDIO_REGISTER");

for (;;) {
        struct uffd_msg msg;
        ssize_t n = read(uffd, &msg, sizeof(msg));  /* blocks until a vCPU faults */

        if (n <= 0)
                break;                  /* FC exited; the guest is gone, so are we */
        if (msg.event != UFFD_EVENT_PAGEFAULT)
                continue;               /* FORK/REMAP/REMOVE events, if enabled */

        /* The kernel reports the faulting address, not the aligned one. */
        unsigned long addr = msg.arg.pagefault.address & ~(r->page_size - 1);
        int is_write = !!(msg.arg.pagefault.flags & UFFD_PAGEFAULT_FLAG_WRITE);

        resolve_fault(uffd, r, addr, is_write);   /* <-- the whole ballgame */
}

The two ioctls, and what they do to the page table

`UFFDIO_COPY` takes a destination address in the registered range, a source address in your own process, and a length. The kernel allocates a page, copies your bytes into it, installs a present PTE, and wakes the faulting thread. You have committed one page of physical memory, charged to the process and to its cgroup, and you paid a `memcpy` for it. Perfectly reasonable when the page contains something.

`UFFDIO_ZEROPAGE` takes a range and no source at all. On a private anonymous mapping the kernel installs a read-only PTE pointing at the global shared zero page — a single physical page that every process on the machine already shares. No allocation. No copy. Nothing charged. Every zero-resolved page in every guest on the host points at the same physical frame, which is exactly as absurd and exactly as correct as it sounds.

The magic is what happens next. Firecracker's guest memory is a `MAP_PRIVATE` mapping, so when the guest eventually writes to one of those addresses, the write hits a read-only PTE and takes an ordinary copy-on-write fault inside the kernel: allocate a real page, zero it, swap the PTE, done. That is a normal minor fault, not a userfaultfd event — MISSING mode only fires when the PTE is *absent*, and after a `UFFDIO_ZEROPAGE` it is present. Your handler never sees it, never blocks on it, and never has to be correct about it.

So the whole difference reduces to one sentence: `UFFDIO_COPY` commits physical memory the first time a page is *touched*; `UFFDIO_ZEROPAGE` commits it the first time a page is *written*. For a guest that boots and then sits waiting for work, the gap between those two sets is most of its RAM.

Side by side

• Physical memory — UFFDIO_COPY: allocates one anonymous page per fault, charged to the process and its cgroup. UFFDIO_ZEROPAGE: installs a read-only PTE to the shared zero page; nothing is allocated and nothing is charged.
• Data movement — UFFDIO_COPY: a memcpy from your buffer, plus whatever it cost you to obtain those bytes (page-cache read, or an HTTP Range GET on a streaming restore). UFFDIO_ZEROPAGE: no source, no copy, no fetch.
• Struct — UFFDIO_COPY: struct uffdio_copy { dst, src, len, mode, copy }. UFFDIO_ZEROPAGE: struct uffdio_zeropage { range{start,len}, mode, zeropage } — note there is no src field, because there is nothing to read.
• When RAM becomes real — UFFDIO_COPY: at first touch, read or write. UFFDIO_ZEROPAGE: at first write, via an ordinary kernel CoW fault your handler never sees.
• hugetlbfs (2 MiB pages) — UFFDIO_COPY: supported, and the only option. UFFDIO_ZEROPAGE: returns EINVAL; there is no shared huge zero page to install through this interface.
• Correctness risk — UFFDIO_COPY: low; you copied the actual snapshot bytes. UFFDIO_ZEROPAGE: depends entirely on your zero-map being right — a stale one hands the guest zeros where real data lived, and that is silent corruption, not an error.
• What it buys — UFFDIO_COPY: nothing you weren't already getting. UFFDIO_ZEROPAGE: sandboxes per host. It is a density optimisation, not a latency one.

You need a zero-map, and it has to be baked

To take the cheap branch you have to know the page is zero, and the obvious way to find out — look at the bytes — is precisely the cost you were trying to avoid. On a streaming restore, where the memory file lives in object storage, that means a network round trip to discover there were no bytes worth having. Even on local disk it's a page-cache read plus a 4096-byte comparison, executed while a vCPU sits blocked. You cannot afford to ask the question at fault time.

So you answer it once, at bake time. Scan the memory file when you create the snapshot and emit a sidecar bitmap recording which chunks are non-zero. Ours is a small header next to the memory image with its own magic (`PSM1`), and at fault time the decision collapses to a bit test on a mapped array. The bitmap does double duty: the prefetcher knows not to warm chunks that hold nothing, and the chunk cache knows not to store them. Absence is the cheapest data you will ever ship.

/* One fault. The only question that matters is whether this page holds
 * anything -- and we answer it from the baked zero-map, never by reading
 * the snapshot bytes, because reading them is the cost we're avoiding. */
static void resolve_fault(int uffd, struct region *r,
                          unsigned long addr, int is_write)
{
        size_t page  = r->page_size;          /* 4096, or 2 MiB on hugetlbfs */
        off_t  off   = r->file_offset + (addr - (unsigned long)r->base);
        int    empty = zeromap_is_zero(r->zeromap, off, page);

        /* Take the free branch only for a read fault on a 4 KiB VMA. A write
         * fault is going to force the CoW allocation immediately anyway, and
         * hugetlbfs rejects UFFDIO_ZEROPAGE outright (see below). */
        if (empty && !is_write && !r->hugepages) {
                struct uffdio_zeropage zp = {
                        .range = { .start = addr, .len = page },
                        .mode  = 0,           /* 0 == install AND wake the vCPU */
                };
                if (ioctl(uffd, UFFDIO_ZEROPAGE, &zp) == 0)
                        return;               /* mapped the shared zero page: 0 RAM */
                if (errno == EEXIST)
                        return;               /* another vCPU already won this page */
                if (errno != EINVAL)
                        die("UFFDIO_ZEROPAGE");
                /* EINVAL: this VMA doesn't support it. Fall through to COPY. */
        }

        /* Non-zero page, write fault, or hugepages: materialise real bytes.
         * For an empty page we copy from a preallocated zero buffer rather
         * than fetching anything -- same PTE result, no I/O. */
        void *src = empty ? r->zero_buf
                          : chunk_fetch(r, off, page);   /* cache, then Range GET */

        struct uffdio_copy cp = {
                .dst  = addr,
                .src  = (unsigned long)src,
                .len  = page,
                .mode = 0,
        };
        if (ioctl(uffd, UFFDIO_COPY, &cp) < 0 && errno != EEXIST)
                die("UFFDIO_COPY");           /* EEXIST is a race, not a failure */
}

Hugepages: the optimisation that eats this one

Backing guest memory with 2 MiB hugetlbfs pages is the other big lever here, and it's a good one: one fault covers 2 MiB instead of 4 KiB, so you take on the order of 512× fewer faults and the restore's tail shrinks with them. If you're latency-bound, reach for it first.

It also takes `UFFDIO_ZEROPAGE` away from you. The ioctl returns `EINVAL` on a hugetlbfs VMA — there is no shared huge zero page for the kernel to install through this interface, so a hugepage guest must answer every single fault with `UFFDIO_COPY`, zeros included. And the damage compounds: with a 2 MiB granule, a single non-zero byte anywhere in the chunk marks the whole 2 MiB as non-empty, so even a hypothetical huge-zero path would fire far less often than the 4 KiB one does.

These two optimisations do not compose, and pretending otherwise wastes a week. Pick based on which resource you run out of first. On our fleet, hosts hit their memory ceiling long before they hit a restore-latency ceiling — create is p50 179ms / p99 203ms with the restore step itself around 49ms, and nobody has ever asked me to make that faster — so the density branch wins for general sandbox templates. There's an operational catch too: hugepage-ness is a property of the snapshot, not the runtime, so a hugepage guest's snapshot can only ever be restored through the UFFD path, and flipping the flag means re-baking every template.

Write faults, EEXIST, and other ways to wedge a vCPU

Two details in that handler deserve more than a comment. The first is `UFFD_PAGEFAULT_FLAG_WRITE`. If the faulting access is a write and the page is empty, `UFFDIO_ZEROPAGE` is correct but guarantees an immediate second fault: you install a read-only zero mapping and the guest's next instruction takes the CoW. That second fault is cheap — entirely inside the kernel, no userspace round trip — so both choices are defensible. I answer write faults with `UFFDIO_COPY` from a preallocated zero buffer, because you pay for the page either way and one fault beats two. The effect is small; measure it before believing me.

The second is `EEXIST`, and this one bites people. Two vCPUs can fault the same page at nearly the same moment, or a background prefetch thread can install a chunk a microsecond before the fault handler gets to it. The kernel serialises the installation, and the loser's ioctl comes back `EEXIST` — one or more pages in the range were already mapped. That is not an error. The page is present, the blocked thread has already been woken by whoever won, and the correct handling is to return successfully and move on. Do not retry it. Do not abort the handler. I have seen more than one implementation treat `EEXIST` as fatal, which converts a benign race into a dead handler, which converts a dead handler into a guest that is frozen rather than crashed.

• EEXIST — page already mapped by a racing vCPU or your own prefetcher. Treat as success; the faulting thread is already awake.
• EINVAL on ZEROPAGE — the VMA doesn't support it (hugetlbfs is the case you'll hit). Fall through to COPY rather than failing the fault.
• EAGAIN — the mapping changed underneath you, usually because the VM is being torn down. Re-read the region set, or exit cleanly if Firecracker is gone.
• read() returning 0 or EOF on the uffd — Firecracker exited. Stop; don't spin. Anything else and you'll burn a core per dead guest.
• Any unhandled error — never let the process die with faults outstanding. An unanswered fault is not a crash, it's a permanently blocked vCPU, and it will be reported to you as "the sandbox is slow".

How to actually see the difference

You cannot see this in a latency histogram, so don't look there. Watch what the guest's memory costs the host: anonymous RSS of the Firecracker process, or better, `memory.current` of the cgroup you put the VM in, since that's what capacity planning is denominated in. Zero-page mappings are present PTEs, but they point at a page the kernel already owns, so they don't inflate the anonymous accounting the way copied pages do.

# Find the Firecracker process backing one sandbox.
pid=$(pgrep -f "firecracker --id $SANDBOX_ID")

# The one-line summary. Rss is what the mapping costs; Anonymous and
# Private_Dirty are the parts that are genuinely private RAM. Pages you
# resolved with UFFDIO_ZEROPAGE do not show up there until they're written.
grep -E '^(Rss|Pss|Anonymous|Private_Dirty):' /proc/$pid/smaps_rollup

# Per-mapping view -- guest RAM is the one enormous rw-p anonymous VMA.
awk '/rw-p/ {v=$0} /^Rss:/ {print v, $2}' /proc/$pid/smaps | sort -k2 -n | tail -3

# The number capacity planning actually cares about, if the VM has its own
# cgroup: it counts the CoW page a guest write allocates, not the zero PTE.
cat /sys/fs/cgroup/pandastack/$SANDBOX_ID/memory.current

# Host-wide, across a batch of restores. Sample before and after.
grep -E '^(AnonPages|MemAvailable):' /proc/meminfo

# The honest experiment: same snapshot, same guest workload, two builds of
# the handler (ZEROPAGE branch on / forced off). Restore N sandboxes, let
# each boot to idle, then diff AnonPages. Do NOT diff the boot latency --
# that is not what changed.

The methodology matters more than the commands. Restore a batch, let each guest reach a steady idle state, and compare host anonymous memory between a build that takes the zero branch and one that doesn't — a single guest's difference is smaller than the variance from whatever else the host is doing. Here's the driver, unglamorous on purpose:

from pandastack import Sandbox

# Density probe: restore a batch from the same baked snapshot, push each
# guest through a realistic warm-up, and watch host memory -- not the clock.
# Switching the handler to UFFDIO_ZEROPAGE barely moves create latency
# (p50 179ms / p99 203ms either way). It moves this.

WARMUP = "import json, re, sys\nprint(sum(range(100000)))\n"

boxes = []
try:
    for _ in range(20):
        sbx = Sandbox.create(template="code-interpreter", ttl_seconds=900)
        boxes.append(sbx)

        # A guest that only ever boots looks unrealistically cheap. Make it
        # import a real stack so the working set resembles production.
        sbx.filesystem.write("/work/warmup.py", WARMUP)
        r = sbx.exec("python /work/warmup.py")
        assert r.exit_code == 0, r.stderr

    # Now read /proc/meminfo on the HOST (not in the guest) and compare
    # against the pre-batch sample. The delta in AnonPages divided by 20 is
    # your real per-sandbox memory cost -- the number that decides how many
    # of these fit on a box.
    input("sample host AnonPages now, then press enter to tear down")
finally:
    for sbx in boxes:
        sbx.kill()

This buys density, not speed. Be honest about it.

I want to be very clear, because this is the part that gets oversold. Moving zero faults to `UFFDIO_ZEROPAGE` does not make snapshot restore meaningfully faster. Per fault it is cheaper — no source bytes to obtain, no `memcpy`, no allocation — so if you were making network round trips for chunks that turned out to be empty, the tail can improve a little. But create on PandaStack is p50 179ms / p99 203ms with the restore step around 49ms, and that budget is dominated by things unrelated to this branch: process spawn, device setup, network attach, guest resume.

What it changes is the denominator of your capacity math. With `UFFDIO_COPY` everywhere, a restored guest's private memory grows toward *every page it has ever touched*. With the zero branch, it grows toward *every page it has ever written*. When we looked at where restore-time faults were actually going on our fleet, the large majority of them were zero-fills being held as private anonymous RAM — pages the guest read once, never wrote, and would happily have shared with every other guest on the machine. Moving those to `UFFDIO_ZEROPAGE` was a density win. Sandboxes per host went up; the stopwatch didn't move.

And one more piece of honesty: this is a deferral, not an abolition. You are leaning harder on the empirical fact that guests don't write most of what they read. That's overcommit with better manners, and it's fine right up until a workload changes shape and starts dirtying pages it used to only read — at which point the memory becomes real, all at once, on a host you sized assuming it wouldn't. Keep headroom for some fraction converting, and keep watching `memory.current` rather than your own model of it.

When this is overkill, and what it costs you

If you're not running a userfaultfd handler at all, don't build one for this. Firecracker's ordinary restore path mmaps the memory file `MAP_PRIVATE` and lets the kernel demand-page it from the page cache, and if the snapshot file is sparse the kernel's own handling of holes gives you much of this for free — reads of a hole map the zero page, only writes allocate. A UFFD handler earns its complexity when memory has to come from somewhere the kernel cannot mmap, which for us means object storage. If your snapshots live on local disk, you have a simpler and better-tested path available and I'd take it.

If you run one or two VMs per host, density isn't your constraint and this is a rounding error on a machine you've already bought. If you've chosen hugepages for tail-latency reasons, you can't have it at all. And if your guests are memory-hungry by design — a database, a compiler, anything that fills its RAM on purpose — the zero-map simply won't fire very often, because there genuinely isn't much nothing to find. Our managed Postgres VMs behave nothing like an idle agent sandbox, and I'd be lying if I claimed the same win applies there.

The costs are real and worth stating plainly. You add a bake-time scan of every memory image, and an artifact that must be versioned, shipped, content-addressed, and invalidated in lockstep with the snapshot — a new way for a deploy to be subtly wrong. You add a branch to the hottest loop you own, where every microsecond is charged to a blocked vCPU. And you introduce one genuinely nasty failure mode: a stale zero-map corrupts guest memory silently, with no exception, no log line, and a symptom that surfaces somewhere else entirely. Write the test that restores a guest, checksums a known non-zero region against the snapshot bytes, and fails loudly — before you ship, not after.

With those guardrails, though, it's one of the better trades available in this part of the stack. A handful of lines in a fault handler, a bitmap nobody ever looks at, and the physical memory you stop buying to store nothing at all.