Snapshot-Restore vs Live Migration: Not the Same Problem

"Doesn't Firecracker already do live migration? It has snapshot/restore." It's a fair question to ask, and the honest answer is that snapshot-restore and live migration both involve freezing a virtual machine's state and picking it up somewhere else — but that surface similarity hides two engineering goals that pull in almost opposite directions. Live migration exists to keep ONE VM continuously available while you move it off a host — a maintenance and load-balancing primitive, where the guest's uptime, connections, and identity are supposed to survive the move basically unnoticed. Snapshot-restore, as Firecracker implements it and as PandaStack uses it for every sandbox create, exists to stamp out MANY new VM instances cheaply from one baked template — a fan-out and replication primitive, where each restore is a fresh instance, not a continuation of a running session. This post lays out the mechanics of each, compares them dimension by dimension, and explains why PandaStack's architecture is built around the second one and deliberately not the first.

How live migration works: keep the guest running while you move it

Live migration answers a specific operational question: a host needs to go offline — for hardware maintenance, a patch, decommissioning, or just to rebalance load off a hot node — and there's a VM running on it that a real client depends on. You can't just stop the VM and boot it elsewhere; that's an outage. So the classic approach, generally known as pre-copy migration and used in one form or another by QEMU/KVM's migration support and VMware vMotion, works iteratively while the VM keeps running on the source host.

The general shape, at a conceptual level: the hypervisor starts copying the VM's memory pages to the destination host while the guest is still executing on the source. Because the guest keeps running, it keeps writing to memory, which dirties pages that were already copied — so the hypervisor tracks which pages got dirtied during the copy pass (the general idea is usually called dirty-page tracking) and does another pass to re-send just those pages. Each pass should, ideally, dirty fewer pages than the last, so the delta converges toward something small. Once the remaining dirty set is small enough (or a bound is hit), the source VM is paused very briefly, the last delta plus device/CPU state is copied over, and execution resumes on the destination — the classic final stop-and-copy step. Done well, the pause is short enough that a client mid-connection barely notices. The exact algorithms, thresholds, and guarantees differ by hypervisor and version, so treat the above as the well-known conceptual shape rather than a spec — check QEMU/KVM's or your hypervisor's current documentation for precise behavior.

The point of all that iteration is entirely in service of one thing: minimizing the pause for a single VM's single move. Nobody is trying to create a second VM here — there's exactly one guest identity before and after, same IP typically, same running processes, same open connections, just relocated onto different physical hardware. That's why host maintenance and load-balancing are the canonical triggers for live migration — it's a mobility feature, not a creation feature.

How snapshot-restore works: freeze once, restore as many new instances as you want

Snapshot-restore starts from a completely different question: how do you create a new, fully-booted VM in well under a second instead of paying a multi-second kernel boot every time? Firecracker's answer is not iterative at all. You boot a guest once, let it finish initializing, then freeze it at a single point in time — the guest is briefly paused while the hypervisor dumps its physical memory (vm.mem) and its device/vCPU state (vm.state) to files. That's it. One freeze, no dirty-page convergence loop, no destination host waiting on the other end of a copy stream.

The payoff comes afterward, and it's where the two techniques diverge hardest: that one snapshot can be restored independently, as many times as you want, on any capable host. Restoring maps the frozen memory back in (copy-on-write, so pages are shared until a guest writes to them) and resumes the vCPUs from the saved state — the new guest picks up mid-instruction with a warm page cache and no init sequence, because it's not booting, it's resuming a machine that already finished booting once, at bake time. Each restore is an independent new instance. Nobody is trying to keep "the same VM" alive across the operation — the whole design center is to let one baked template answer many separate create requests, which makes it a fan-out and replication primitive rather than a mobility primitive. PandaStack leans on exactly this for every sandbox create: a restore step lands around a 179ms p50 (roughly 203ms p99), against a first-ever cold boot of a template at around 3 seconds before that snapshot exists to restore from.

The comparison, dimension by dimension

Once you separate mobility from fan-out, most of the apparent overlap resolves into contrast.

• Primary goal — Live migration: keep one running VM available while relocating it off its current host. Snapshot-restore: create many new VM instances cheaply from one baked template.
• Guest identity/continuity — Live migration: the guest's identity, uptime, and running state are meant to persist across the move — it's still considered the same VM. Snapshot-restore: each restore is a fresh instance from the guest's perspective; there's no single continuous session being preserved across restores.
• Network connection preservation — Live migration: typically aims to keep live client connections working post-move, often relying on the same subnet/L2 domain or careful re-homing so the IP keeps resolving. Snapshot-restore: does not try to preserve a live client connection across a restore — PandaStack's guests wake up with the network identity (IP/MAC/gateway) frozen at bake time and patched onto a fresh tap, which is a template's NATID slot identity staying consistent across restores, not one client's live TCP session continuing.
• Iterative vs single-shot — Live migration: iterative pre-copy passes with dirty-page tracking, converging toward a short final stop-and-copy. Snapshot-restore: a single point-in-time freeze, done once, followed by any number of independent, non-iterative restores later.
• Typical trigger — Live migration: host maintenance, hardware issues, or load rebalancing on infrastructure the operator controls. Snapshot-restore: a new-instance creation request — every sandbox create on PandaStack takes this path, not just occasional operational events.
• What "downtime" means — Live migration: downtime is the brief pause during final stop-and-copy for the one VM being moved — the metric operators optimize is how short that pause is. Snapshot-restore: there's no VM being kept alive to have downtime; the closest analog is create latency for a brand new instance, not an interruption to something already running.
• Relationship to guest count — Live migration: 1:1 — exactly one guest before and after the operation, just on different hardware. Snapshot-restore: 1:many — one frozen template can back an arbitrary number of independently-created guests.
• Underlying primitive — Live migration: iterative memory copy plus a final handoff, engineered around minimizing guest-visible interruption. Snapshot-restore: a single freeze-to-disk plus repeatable, independent restores, engineered around minimizing per-create latency and idle cost.
• Best fit — Live migration: long-lived VMs that a real workload depends on staying up, where the operator needs to drain a host without an outage. Snapshot-restore: ephemeral, disposable-by-default instances where the goal is spinning up a lot of fresh, identical starting points fast.

Does Firecracker do traditional live migration?

This is the question worth being careful about instead of confidently wrong about. Firecracker's public design center and documented feature set is centered on snapshot/restore as a fast-boot and create-density primitive — freeze once, restore many, minimize per-create latency. That is not the same feature set as the pre-copy, iterative, dirty-page-tracking live migration associated with full hypervisors like QEMU/KVM's own migration support or VMware vMotion, where the entire point is keeping one specific running guest alive and reachable across a host move. Whether Firecracker has, lacks, or is moving toward some form of that traditional live-migration capability isn't something this post is going to assert either way — check Firecracker's current documentation and roadmap directly if that specific capability matters for your use case.

The architectural point that survives regardless of the exact feature-flag answer is this: even in places where snapshot/restore-style mechanics show up in both worlds — freezing memory and state to files, mapping them back in on the other side — the engineering goal each system is optimizing for is different. One is optimizing to move a single continuous instance without dropping it. The other is optimizing to spin up many new, independent instances as fast and cheaply as possible. Those goals produce different guarantees, different failure modes, and different things you're allowed to assume about the guest before and after — and that's the distinction worth carrying forward, independent of what any particular hypervisor's changelog says this month.

Where cross-host fork fits (related, but still not migration)

PandaStack's fork feature moves a snapshot's artifacts to a different agent host and restores a new, independent sandbox there — same-host forks land in 400–750ms, cross-host forks in 1.2–3.5s once the memory and rootfs artifacts travel over the network. On the surface that looks close to live migration, because state is genuinely moving between hosts. But the intent is the same fan-out intent as any other restore: a cross-host fork spawns a child instance elsewhere, and the original sandbox keeps running right where it was. Nothing about the original VM's identity, connections, or uptime is being relocated — you end up with two independent sandboxes, parent and child, not one VM that moved. That's fan-out with a network hop in it, not migration.

Why PandaStack's architecture doesn't try to do live migration

PandaStack's whole create path is built on a deliberate design decision described plainly in its own architecture: there is no warm pool of idle VMs. Every sandbox create restores a baked template snapshot on demand through the NATID fast path rather than handing out a VM that's already been sitting around running. That single decision is also what makes live migration a much less relevant primitive here than it is for a fleet of long-lived, individually-important VMs.

Live migration earns its complexity when a specific running VM matters enough that losing it, even briefly, is a problem you engineer around — that's the world of a database VM or a customer-facing service instance that's been up for weeks and has state and connections nobody wants to drop. PandaStack's sandboxes are the opposite of that by design: short-lived, disposable, and cheap to recreate. If an agent host needs to be drained for maintenance, the answer isn't to iteratively pre-copy a running sandbox's memory to a new host without dropping it — it's to let that ephemeral sandbox end and have the next create (or a fork, if continuity of a particular filesystem/memory state actually matters) land on a healthy host in a couple hundred milliseconds. Managed databases are the one place PandaStack does treat instances as longer-lived and pins them to a host, and even there the reliability answer is failover — rebuild from the durable GCS archive on a healthy agent — rather than a live, in-flight relocation of a running VM.

Put differently: live migration is a tool for protecting the uptime of things you don't want to lose. Snapshot-restore is a tool for making it cheap not to care if any one instance goes away, because a fresh one is a sub-second restore away. Building a system around "nothing here is precious enough to need protecting in place" is precisely what makes the fast, iteration-free, single-freeze-many-restores model the right fit — and what makes the iterative, one-VM-at-a-time machinery of live migration mostly unnecessary overhead for this workload.

The summary

Live migration and snapshot-restore both freeze VM state and pick it up somewhere else, which is exactly why they get conflated — but they're solving different problems at different cardinalities. Live migration is a 1:1 mobility primitive: keep one running VM's connections and identity alive across an iterative pre-copy, dirty-page-tracking, stop-and-copy move to a new host, typically for maintenance or load balancing. Snapshot-restore is a 1:many fan-out primitive: freeze a booted guest once, then restore that exact machine independently as many times as you need new instances, with each restore a fresh instance rather than a continued session. Firecracker's documented design center is squarely the second one — verify against its current docs if you need the first — and PandaStack builds its entire sub-second create path on it precisely because its sandboxes are ephemeral by design, with no warm pool of long-lived VMs whose uptime would ever need the protection live migration exists to provide. The core is open source under Apache-2.0, so the restore path — NATID networking, copy-on-write memory, the whole create pipeline — is something you can read and run yourself rather than take on faith. For the full create pipeline, see /blog/snapshot-restore-boot-path; for the memory mechanics, /blog/firecracker-memory-snapshots; for how snapshot-restore compares to the closest checkpoint/restore cousin at the process level, /blog/snapshot-restore-vs-criu-checkpoint.