The Serverless Cold-Start Problem, Explained

Every serverless platform makes the same promise — scale to zero, pay only for what you run — and every one of them ships with the same asterisk: the cold start. It's the pause between "a request arrived" and "code is running," and it's the single most-complained-about property of serverless, because it lands exactly where users notice it: the first request after a quiet period, the tail latency of a bursty API, the agent that hangs for two seconds before it does anything. The frustrating part is that a cold start isn't one problem you can fix with one flag. It's a stack of costs, each with its own cause, and the popular mitigations each attack a different layer at a different price. This is the explainer: what a cold start is actually made of, where each millisecond goes, and an honest accounting of every standard fix — including the newer one, snapshot-restore, and where it gets genuinely hard.

What a cold start actually is

"Cold start" gets used as if it were a single event, but on a serverless platform it's a pipeline of distinct stages that all have to complete before your handler runs for the first time. When people quote a scary cold-start number, they're quoting the sum of all of them. Pull them apart and you can see which mitigations actually touch which cost — and which are just papering over the whole thing.

1. Provision the execution environment — the platform has to find a host with capacity and stand up an isolated sandbox to run your code in: a container, a microVM, or a V8 isolate. On a microVM this is VMM launch, disk setup, and network plumbing; on a container it's image pull and namespace/cgroup setup. Milliseconds to seconds depending on the isolation model and whether the image is already local.
2. Initialize the language runtime — start the interpreter or VM the code needs. A JVM has to spin up, load core classes, and JIT-warm; a Python interpreter has to start and import its startup modules; a Node process has to boot V8 and its module system. This is pure per-runtime overhead before a single line of your code runs, and it's why the same function is cheaper on a lighter runtime.
3. Load dependencies — import your libraries. This is often the biggest surprise: a function that imports boto3, pandas, or a fat framework can spend hundreds of milliseconds just resolving and loading modules off disk. The bill scales with your dependency tree, not your logic.
4. Run application init — the code that runs once per environment before the handler: reading config, building a DB connection pool, warming an in-memory cache, constructing an SDK client. Anything you put at module scope or in a constructor lands here.
5. Set up the network — attach the sandbox to a network, get it an address, and (the expensive part) establish outbound connections: DNS, TLS handshakes to your database and downstream services. A fresh TLS handshake plus connection-pool warm-up can quietly cost more than the runtime init.

The key insight is that these stack serially, and a warm invocation skips almost all of them — it reuses an already-provisioned environment whose runtime is running, dependencies are loaded, app init has completed, and connections are open. So a cold start is really the cost of everything a warm environment already did once. Every mitigation is, at bottom, a different answer to the question: how do we avoid paying for this stack on the request that a user is waiting on?

The standard mitigations, and what each really costs

There are four broad families of fix in wide use. Each targets a different part of the stack, and — this is the part the marketing pages skip — each has a real cost that's usually paid somewhere the latency graph doesn't show. Here's the honest comparison.

• Warm pool / provisioned concurrency — Pro: near-zero cold starts for provisioned capacity; the environment is already booted, runtime running, deps loaded, so a request just gets handed a ready worker. Con: you pay for that capacity continuously whether or not a request ever arrives — it's paying to keep the lights on in an empty building. You also have to guess how much to provision: too little and a burst overflows into real cold starts at the worst moment; too much and you're burning money on idle. It defeats the whole 'scale to zero' premise.
• Keep-alive pings — Pro: cheap and hacky; a scheduled ping every few minutes keeps one environment from being reaped, so the next real request finds it warm. Con: it's a hack, not a fix. It keeps exactly one (or a handful of) environments warm, so it does nothing for concurrency — a second simultaneous request still cold-starts. It's fighting the platform's own idle-reaping heuristics, breaks silently when they change, and quietly bills you for invocations whose only purpose is to lie to the reaper.
• Lighter runtime — Pro: attacks stages 2 and 3 at the source. Swapping a JVM for Go or a compiled binary, or trimming your dependency tree, genuinely shrinks the cold start rather than hiding it. This is often the highest-leverage change and it's free of ongoing cost. Con: it's a rewrite, not a config toggle; you can't always choose your runtime; and it caps out — a lighter runtime makes the cold start smaller but never zero, and it does nothing for provisioning or connection setup.
• Snapshot-restore — Pro: skips stages 2 through 4 entirely by capturing an environment after it's fully initialized — runtime running, deps loaded, app init done — and restoring that frozen state per request instead of rebuilding it. Turns a multi-second init into a tens-of-milliseconds restore, with no idle fleet to pay for. Con: a snapshot is a frozen moment, which creates real correctness gotchas — stale state, entropy reuse, clock skew, dead connections (covered below) — and it needs a substrate that can snapshot and restore cheaply, which not every platform has.

Notice the pattern. Warm pools and keep-alive pings don't make the cold start faster — they pay, continuously, to make sure you rarely hit one. A lighter runtime makes it genuinely smaller but can't reach zero. Snapshot-restore is the only one of the four that removes most of the work rather than paying to avoid it. That's why it's worth understanding both why it wins economically and where it bites.

Why snapshot-restore beats warm pools economically

The economic argument is simple once you see it. A warm pool's bill is set by your provisioned peak and paid 24/7. To absorb a burst you must provision for the burst, which means for most of the day most of the pool is idle — fully-resident environments holding RAM and a scheduled CPU, doing nothing but existing so they can be handed out instantly. You are, quite literally, paying to keep machines warm that no one is using, and you're paying for peak even during the trough.

Snapshot-restore inverts that. There is no idle fleet. You bake a fully-initialized environment into a snapshot once, keep zero warm copies, and on every request restore a fresh copy from that snapshot. The restore is cheap because it isn't a boot — the runtime was already up, the dependencies already loaded, app init already done when the snapshot was frozen; restoring maps that state back and resumes execution mid-flight. So your bill tracks actual concurrent usage, not provisioned peak, and the idle cost rounds to zero because nothing idles. The more bursty your traffic — and agent, CI, and per-request workloads are almost always bursty — the wider the gap between 'pay for peak, continuously' and 'pay for what's actually running.'

A warm pool solves cold starts by never letting the machine go cold — which is also the bill. Snapshot-restore lets everything go cold and restores it faster than a warm pool can hand one out. One pays to avoid the work; the other skips the work.

This is the idea behind the newer platform-level approaches. AWS Lambda SnapStart, for Java and some other runtimes, boots your function once, runs its init, takes a Firecracker microVM snapshot of the initialized state, and restores that snapshot on subsequent cold starts instead of re-running init — collapsing the JVM-and-init cost that dominated Java cold starts. Firecracker's own snapshot/restore is the primitive underneath, and it's what general-purpose sandbox platforms build on to restore a fully-warm VM per create. The specifics — which runtimes, what's charged for the cached snapshot, exact latencies — differ by provider and change over time, so verify the numbers against their docs; the shape of the idea is the same everywhere: freeze after init, restore instead of re-init.

Where snapshot-restore gets hard: the gotchas

None of this is free, and pretending otherwise is how you ship a subtle production bug. A snapshot is a photograph of a running process at one instant, and restoring it means many copies of that exact instant now run in parallel and into the future. Four categories of thing that were fine at freeze-time can be wrong at restore-time, and every serious snapshot-restore system has to answer all four.

State freshness

Anything your init cached is frozen at bake-time values: a config blob, a feature-flag set, a JWT signing key, a warmed in-memory cache. Restore it a day later and the handler is serving stale data it thinks is current. The fix is discipline about what you put before the snapshot line — cache only what's genuinely immutable, and re-fetch anything that can change on the warm path rather than at module scope. SnapStart formalizes this with runtime hooks so you can invalidate or refresh state right after restore.

Entropy and RNG

This is the sharp one. If you seed a random number generator during init and then snapshot, every restored copy shares the same seed — and now N "independent" environments generate the identical sequence of "random" numbers. That's a real, exploitable security bug when the RNG is producing session tokens, nonces, or key material. The mitigation is to re-seed from a fresh entropy source after restore, never before the snapshot; well-designed platforms reinitialize the guest's randomness on resume so restored copies diverge.

Clock skew

A snapshot freezes the guest's clock. When you restore hours or days later, the guest can wake believing it's still bake-time until something corrects it — and that's not cosmetic. TLS certificate validation, token expiry checks, and anything time-based can misfire: a guest whose clock is stuck in the past may reject freshly-issued certs or accept expired ones. The fix is an explicit clock-sync on restore, so the guest's notion of 'now' is corrected the instant it resumes rather than drifting until the next NTP tick.

Connection staleness

If your init opened a database connection pool or a long-lived socket and you snapshot with those open, every restored copy inherits file descriptors pointing at connections that are long dead — the TCP session on the other end timed out ages ago. The first query hangs or errors. The rule is the same as for state: establish connections lazily on the warm path, not at module scope before the snapshot, or explicitly tear down and rebuild them in a post-restore hook. A frozen connection is worse than no connection because the code thinks it has a live one.

How PandaStack frames it: restore on every create, no warm pool

PandaStack takes the snapshot-restore bet all the way down: there is no warm pool of idle VMs at all. Every sandbox create restores a baked Firecracker microVM snapshot on demand — a fully-initialized guest, runtime up and reachable — so you pay a per-create restore step instead of a continuous idle bill. The restore is cheap enough to do on every single create because it maps the snapshot's memory copy-on-write and pages it in lazily, so a create only materializes the working set the guest actually touches, not the whole image. The result is warm-start-class latency with an idle cost that rounds to zero.

from pandastack import Sandbox

# create() restores a baked template snapshot on demand.
# No warm pool was consulted; nothing was kept idling for you.
# ~179ms p50, ~203ms p99 -- warm-start-class, with no standing bill.
box = Sandbox.create(template="code-interpreter")

# The guest is already booted and reachable -- run immediately,
# no "wait for ready" loop (the readiness probe is part of create).
result = box.exec("python -c 'print(2 ** 10)'")
print(result.stdout)  # -> 1024

# Kill it. Between now and the next create, NO VM of yours is
# parked on a host burning RAM. You pay for sandboxes that exist,
# not for a pool that might be needed.
box.delete()

The honest asterisk is the one every snapshot system has: you need a snapshot to restore. The very first spawn of a brand-new template does a real ~3s cold boot to reach ready, then captures the snapshot (auto-bake); every create after that takes the ~179ms p50 / ~203ms p99 restore path. And the four gotchas above are handled on the substrate rather than left to you — the guest's clock is corrected on restore/resume/wake so freshly-rotated TLS certs don't break, and the restore path re-establishes the environment-specific state that would otherwise be frozen. The same primitive powers forking a live sandbox (same-host 400–750ms, cross-host 1.2–3.5s as the artifacts cross the network) and managed Postgres databases (30–90s, because a database bootstrap is genuine per-instance work no snapshot can skip). Each agent pre-builds up to 16,384 /30 network subnets so the plumbing is never on the hot path either.

So the cold-start problem, in one frame: it's a serial stack of provision, runtime-init, dependency-load, app-init, and network setup, and your options are to pay continuously to keep it warm (warm pools, keep-alive pings), shrink it at the source (lighter runtimes), or skip most of it by freezing an already-warm environment and restoring it (snapshot-restore) — the last of which wins economically for bursty traffic but demands you handle state freshness, entropy, clock skew, and connection staleness honestly. PandaStack's core control-plane API and per-host agent are open source under Apache-2.0, so you can run them on your own Linux KVM hosts and watch both the restore timings and the empty idle bill yourself; verify any competitor's numbers against their own docs. For the head-to-head economics see /blog/snapshot-restore-vs-warm-pools, and for the create pipeline that produces those milliseconds see /blog/microvm-cold-start-optimization.