all posts

Cloud Dev Environments on microVMs

Ajay Kumar··9 min read

A cloud development environment is a full, ready-to-code machine that lives in the cloud instead of on your laptop: clone the repo, install the toolchain, run the dev server, attach an editor over SSH or the browser. Codespaces, Gitpod, Coder, and Daytona all sell some version of this. The interesting question isn't whether the idea is good — it clearly is — but what you run it on. A dev environment quietly wants three things that usually trade off against each other: strong isolation (you're running a teammate's branch, and its postinstall scripts, on shared hardware), fast start (nobody waits two minutes to fix a typo), and the ability to freeze and resume (so an idle environment costs nothing but wakes up where you left it). A Firecracker microVM is one of the few substrates that gives you all three at once. This post is about why, and how it maps onto PandaStack's primitives.

What a dev environment actually wants

Strip away the editor integrations and a cloud dev environment is just a sandbox with a long lifetime and write access to a repo. The hard parts are operational. You're going to run code you didn't write — not just the project's source, but every transitive dependency and every `postinstall` hook in `node_modules`, every `pip install` that runs arbitrary `setup.py`, every Makefile target. On a shared host, that's untrusted code by any honest definition. You also want it to feel local: instant to start, snappy to use, and there when you come back tomorrow. Those goals pull in different directions on most infrastructure, which is why so many dev-environment products end up bolting workarounds onto whatever they started with.

  • Isolation — Dev env on a container: shares the host kernel with every other tenant; a malicious or buggy postinstall script is one container-escape away from neighbors. Dev env on a microVM: its own guest kernel under hardware virtualization (KVM); arbitrary build scripts are contained to one disposable VM.
  • Start time — Dev env on a container: fast to start the container, but the slow part is provisioning + installing the toolchain on a cold image. Dev env on a microVM: snapshot-restore makes the VM itself appear in ~49ms (the restore step); the win compounds when the snapshot already has the toolchain baked in.
  • Freeze + resume — Dev env on a container: pause/checkpoint of a full userspace is awkward and rarely exact. Dev env on a microVM: hibernate snapshots memory + disk and stops the VM; the next request wakes it to the exact process state, so an idle env costs nothing.
  • Reproducibility — Dev env on a container: an image gets you a consistent filesystem, but in-flight process state and the configured environment drift. Dev env on a microVM: snapshot a fully configured, post-setup machine once, then fork it — every developer starts from the identical known-good state.
  • Blast radius — Dev env on a container: a runaway build (a 40GB allocation, a fork bomb) can starve the host. Dev env on a microVM: capped to the VM's own memory and the VM's own kernel.
Containers aren't the enemy here — they're a great way to build the image you bake into the environment. The argument is about the runtime boundary: when the thing running inside is a teammate's untrusted branch plus its dependency tree, a shared host kernel is not the boundary you want between tenants.

The isolation angle: postinstall scripts are untrusted code

It's easy to think of a dev environment as trusted because it's running your own company's code. But the moment you `npm install`, you're executing hundreds of packages' lifecycle scripts, most of which you've never read, many maintained by people you've never met. Supply-chain attacks via malicious npm/PyPI packages are a recurring, real category — not hypothetical. If a poisoned dependency runs in a container that shares the host kernel with another developer's environment, the relevant question becomes how good your kernel and your container runtime are at withstanding a hostile local process. With a microVM, the same poisoned package runs against its own guest kernel; to reach a neighbor it would have to escape the hypervisor itself, a far smaller and more heavily audited surface than the full Linux syscall interface. This is the same property that lets AWS Lambda run untrusted functions from thousands of customers on shared fleets — on Firecracker, the VMM PandaStack is built on.

The practical upshot: you can give every developer (or every pull request) its own environment without auditing the dependency tree first, because the isolation boundary doesn't depend on the code inside behaving.

Per-developer and per-branch ephemeral environments

Once an environment is cheap and isolated, the natural pattern is to make it ephemeral: one per developer, or one per branch, or one per pull request, created on demand and thrown away when done. "Works on my machine" stops being a sentence anyone says, because there is no my machine — there's a fresh environment built from the same definition every time. A reviewer opens a colleague's branch in a clean environment with the exact toolchain pinned by the repo, runs it, and closes it; nothing leaks into the next review. Here's the minimal shape with the Python SDK: create an environment, drop a setup script in, and exec against it.

from pandastack import Sandbox

# One ephemeral dev environment for a single branch review.
# `base` is the language-agnostic apps runtime (Node, Python, Go, Bun via mise).
with Sandbox.create(template="base", ttl_seconds=3600) as env:
    # Drop a setup script into the guest and run it.
    env.filesystem.write("/workspace/setup.sh", (
        "set -euo pipefail\n"
        "cd /workspace\n"
        "git clone --depth 1 -b feature/login https://github.com/acme/webapp.git\n"
        "cd webapp\n"
        "mise install            # honours .nvmrc / .tool-versions\n"
        "npm ci\n"
        "npm run build\n"
    ))
    result = env.exec("bash /workspace/setup.sh", timeout_seconds=600)
    print("exit:", result.exit_code)
    print(result.stdout[-2000:])  # tail of the build log

    # The reviewer's environment is now live; run tests against it.
    tests = env.exec("cd /workspace/webapp && npm test", timeout_seconds=300)
    print("tests exit:", tests.exit_code)
# environment is destroyed here — nothing leaks into the next review

The `ttl_seconds` is your safety net: an environment a developer forgets to close reaps itself instead of billing forever. For an interactive session you'd keep it alive (more on that below), but for a scripted "spin up, run the branch, report back" flow the context manager kills it on exit and you're done.

Snapshot the configured environment once, then fork it

Cloning a repo and running a full toolchain install on every create is wasteful when most of that work is identical across developers. The better pattern: do the expensive setup once — clone, install dependencies, warm the build cache, configure the editor — then snapshot that fully configured machine. Every new environment is a fork of that snapshot, so each developer starts from the identical post-setup state in a fraction of the time. A same-host fork is 400–750ms and shares memory copy-on-write, so a hundred developers forking the same golden environment cost far less memory than a hundred independent ones — they share the read-only pages until they write. (A cross-host fork is 1.2–3.5s when the parent's snapshot has to be pulled from object storage first.)

This is also the cleanest answer to reproducibility. An image gives you a consistent filesystem; a memory+disk snapshot gives you a consistent running machine — the same warmed caches, the same configured services, the same everything. The bash you'd run once to produce that golden state looks like an ordinary setup, just performed inside the environment you intend to snapshot:

# Run ONCE inside the "golden" environment, then snapshot it.
set -euo pipefail
cd /workspace

# Clone the repo + a stable base ref everyone branches from.
git clone https://github.com/acme/webapp.git
cd webapp

# Pin + install the toolchain the repo declares (mise reads .tool-versions).
mise install

# Install dependencies and warm the build cache so forks start hot.
npm ci
npm run build

# Pre-pull anything else developers always need.
npm run prefetch:fixtures || true

echo "golden environment ready — snapshot this VM, then fork per developer"
Re-baking the golden snapshot is the moment to refresh dependencies, not the per-fork path. Keep secrets and per-developer credentials OUT of the golden snapshot — inject them at fork time. A snapshot freezes everything in memory, including anything you wouldn't want copied to every developer's environment.

Hibernate idle environments to scale to zero

The dirty secret of cloud dev environments is that they're idle most of the time. A developer codes in bursts: an hour of activity, then a meeting, then lunch, then more code. Paying for a running VM through all the gaps is how cloud-dev-environment bills get scary. The fix is hibernate: when an environment goes idle, snapshot its memory and disk and stop the VM — at which point it consumes no CPU and no RAM, just some storage. The next request wakes it back to the exact process state it was in, dev server and all, fast enough that the developer barely notices the gap.

from pandastack import Sandbox

# A long-lived environment for one developer's interactive session.
env = Sandbox.create(template="base", persistent=True)
try:
    env.exec("cd /workspace && git clone https://github.com/acme/webapp.git",
             timeout_seconds=120)
    # ... developer works, dev server running on a port ...

    # Going idle? Freeze it. Memory + disk are snapshotted; the VM stops.
    # Cost drops to storage only; the next request auto-wakes it.
    env.hibernate()

    # ... hours later, the next request transparently resumes the VM
    #     to the exact state — running processes intact.
finally:
    # When the session is truly over, reclaim everything.
    env.kill()

Because the create/restore path is cheap to begin with (a fresh create is p50 179ms and p99 ~203ms via snapshot-restore; a true cold boot is only ~3s on first spawn), waking from hibernate is in the same neighborhood rather than the cold-boot-plus-reinstall cliff you'd hit re-provisioning a container from scratch. The economic shape is the one you want: pay for active development, pay near-zero for idle environments, and never make a developer re-clone and re-install just because they stepped away.

Where this fits — and where it doesn't

A microVM substrate is the right call when environments are multi-tenant (many developers, many branches, shared hardware), when the code inside is effectively untrusted (it is, the moment you install dependencies), and when you care about cost at idle (you do, the moment you have more than a handful of environments). Capacity isn't the bottleneck people fear: an agent pre-allocates 16,384 networking slots, so the practical ceiling is host memory and CPU, not networking — and hibernate keeps the memory bill honest. When the workload is the opposite — a single trusted environment you keep running and fully control — a plain container or a long-lived VM is simpler, and you should use it. But for the Codespaces/Gitpod/Coder/Daytona shape, where the whole point is many cheap, isolated, resumable environments, the microVM's three-way win on isolation, start time, and freeze/resume is exactly the combination the use case is asking for.

Treat any specific behavior of Gitpod, Codespaces, Coder, or Daytona as something to verify against their current docs — they each make different substrate and pricing choices, and those evolve. The architectural argument here is about properties, not a feature-by-feature scorecard: a dev environment wants isolation, speed, and the ability to freeze, and a Firecracker microVM is built to give you all three.

Frequently asked questions

Why run cloud dev environments on microVMs instead of containers?

A dev environment runs untrusted code the moment you install dependencies — every npm/pip postinstall script executes arbitrary code you didn't write. On a container that shares the host kernel, a poisoned dependency is one escape away from other tenants. A Firecracker microVM gives each environment its own guest kernel under hardware virtualization, so build scripts are contained to one disposable VM. You also get fast snapshot-restore start (~49ms restore step) and the ability to hibernate idle environments to near-zero cost — three properties that usually trade off against each other but come together on a microVM.

How do I give every developer or branch its own ephemeral environment?

Create a sandbox per developer, branch, or pull request on demand and throw it away when done. With PandaStack you create on the base template, write a setup script that clones the repo and runs mise install plus your build, exec it, and let the ttl_seconds reap anything left open. Because each environment is its own microVM, you don't need to audit the dependency tree first — the isolation boundary doesn't depend on the code inside behaving. This is what kills 'works on my machine': there is no my machine, just a fresh environment built from the same definition every time.

How does snapshot-and-fork make dev environments reproducible and cheaper?

Do the expensive setup once — clone, install the toolchain, warm the build cache — then snapshot that fully configured machine. Every new environment is a fork of that snapshot, so each developer starts from the identical post-setup running state. A same-host fork is 400–750ms and shares memory copy-on-write, so many developers forking one golden environment cost far less memory than independent ones (cross-host fork is 1.2–3.5s). Unlike an image, a memory+disk snapshot reproduces the running machine — same warmed caches, same configured services — which is the strongest form of 'identical environment for everyone'. Keep secrets out of the golden snapshot and inject them at fork time.

How do I stop idle cloud dev environments from running up the bill?

Hibernate them. Dev environments are idle most of the time — developers code in bursts. When an environment goes idle, call hibernate() to snapshot its memory and disk and stop the VM; it then consumes no CPU and no RAM, only storage. The next request auto-wakes it to the exact process state, dev server and all. Because the restore path is already cheap (p50 179ms create, ~3s cold boot at worst), waking is fast rather than a cold-boot-plus-reinstall cliff. The result is pay-for-active-development with near-zero idle cost, instead of paying for a running VM through every meeting and lunch break.

Run code in a microVM in one API call.

49ms p50 cold start. Fork, snapshot, and scale to zero.

Start free
Written by Ajay Kumar, Founder, PandaStack.