Where Scale-to-Zero Wake Time Actually Goes

Scale-to-zero is an easy promise and a hard product. Anyone can delete an idle app and bring it back later. The question is what the person who shows up during "later" experiences, and that number is where scale-to-zero platforms are actually differentiated.

We shipped true scale-to-zero on PandaStack — sleeping apps free CPU, RAM, and most of their disk, and cost nothing while asleep. Then we spent a program's worth of effort on wake latency, taking it from about 14 seconds to about 1.2. Almost none of that work was in the hypervisor. This is the breakdown, including the two hypotheses I was confident about that measurement destroyed.

The shape of the problem

A wake has to do roughly this: find the sleeping app's saved state, get a VM running from it, make sure the app inside is actually serving, and then route the waiting request to it. Every one of those steps is somewhere latency can hide, and the intuition that the hypervisor step dominates is very strong and very wrong.

Here's a real measured wake from before the work started, on an actual customer app rather than a synthetic test:

• VM restore: 1.4s
• In-guest SSH becomes reachable: +2.1s
• Three sequential commands run inside the guest before any traffic is allowed in: +8.3s
• First proxied request succeeds: +0.7s
• API and CDN overhead: ~1.5s

The VM — the part everyone optimizes, the part with the interesting kernel work — is 1.4 seconds of 14. Roughly nine seconds is orchestration happening on a machine that is already awake and, in many cases, already running the app.

The hypervisor part, and the one real surprise

The VM step did have a genuine bug, and it's a good one because it shows how the same API call can be fast or slow depending on something invisible.

Creating a fresh sandbox loaded a snapshot in about 53ms. Waking a sleeping app loaded a snapshot in about 3,111ms. Same hypervisor, same API call. The difference: a create restores one shared template image that every sandbox on the host reuses, so it's hot in the page cache. A wake restores a private, per-app memory image that nothing else touches, so it's cold — and Firecracker was eagerly populating the whole multi-gigabyte mapping.

Warming the file helped, but only partly. A cold file took 5.9s and a pre-read file still took 3.6s. The remaining cost was paying to map memory the app never touches. The fix was to bake a small header alongside the memory image recording which chunks are actually non-zero, so restore can skip the rest. Guest memory is overwhelmingly zeros — we measured about 6% of chunks as non-zero on a real snapshot.

• Before: 3,111ms to load the snapshot
• After: 33ms

That's a 94x improvement on that phase, and it came from not reading things instead of reading them faster. The same header lets the image be sparsified on disk — punching holes where the zeros were took a real snapshot from 4 GiB allocated down to 256 MiB, which is a storage win that came free with the latency work.

The real cost: doing careful things in the wrong order

With the snapshot load down to 33ms, the profile flipped and the remaining time was all ours. The wake path was doing a sequence of sensible, defensible things — and doing all of them before letting anyone in.

• Polling on a 500ms ticker, which makes 500ms a latency floor regardless of how fast anything else is
• Rewriting the app's environment file, which a process that's already running cannot see anyway
• A disk-coherence probe that called a global filesystem sync — measured at 3.4 seconds, because it was flushing the page cache of everything just restored
• Health checks implemented as commands executed inside the guest, with multi-second timeouts and sleeps between attempts

Every one of those exists for a reason. The disk probe in particular guards against a real incident where an incoherent restore served wrong data for hours. None of them should be deleted. But they were all sitting on the critical path, in front of a user waiting for a page.

The principle: resume, then patch

The fix is a reordering, and the principle is worth stating as a rule: resume first, then patch — never patch, then resume.

A woken app is gated on exactly one fact: does it answer HTTP? Everything else — re-delivering environment variables, verifying disk coherence, confirming SSH — moves behind that gate and runs after the first byte is served. If the coherence check fails afterward, the app is parked in an error state and the router redeploys it. That is a real trade: there's now a window of seconds between serving the first byte and confirming coherence. We took it deliberately, because the alternative was charging every wake several seconds to guard against a rare failure that has a recovery path.

Alongside that, we stopped using SSH readiness as the gate (the app can be serving before SSH is), dropped poll intervals to 150-200ms since a tick interval is a hard latency floor, and made concurrent visitors during a wake hold and forward rather than get bounced to a refresh page.

The measured result, across three real production sleep-wake cycles on a Next.js app:

• Internal wake: 1,381ms, then 1,260ms, then 1,198ms as caches warmed
• First response: about 1.1 seconds
• Visitor-perceived, including DNS, TLS, and CDN: 1.80s and 2.22s, real page, HTTP 200

Down from roughly 14-15 seconds bouncing on a refresh page.

One detail worth stealing

If you gate a wake on "does the app answer HTTP," you need to distinguish your own infrastructure saying the app is down from the app itself returning an error. An app that is itself a proxy will happily emit its own 502, and reading that as "still waking" means waiting forever for something already working.

We classify strictly: only the agent's own error signatures count as down. Everything else, including a 502 the app generated, counts as serving. It's a small piece of code and it's the difference between a gate that works and one that hangs on a whole category of apps.

What's still slow, honestly

Two things we haven't fixed and won't pretend otherwise.

Going to sleep still takes about 50 seconds, dominated by the hypervisor writing a full memory snapshot. Sparsifying reclaims space, not time. Fixing it needs differential snapshots. It's background work, so it's far less user-visible than wake was — but it's not fast.

And sub-one-second visitor-perceived wake isn't reachable while a CDN and TLS handshake sit in front, which is 0.2-0.6s on its own. The internal number can keep improving; the number in a browser has a floor we don't control.

The takeaway

If you're evaluating a scale-to-zero platform, ask what the wake number measures. "Snapshot restore in N milliseconds" is a hypervisor benchmark, not a wake. The number that matters is from request arriving to real page served, measured through whatever CDN sits in front.

And if you're building one: instrument every phase before you optimize any of them. The most valuable thing that came out of this work wasn't any individual fix — it was that measurement contradicted both of my confident hypotheses, twice, and the fixes were in places I would never have looked.