Stateful vs Ephemeral AI Agent Sandboxes

Every AI-agent platform eventually hits the same fork in the road: should the sandbox the agent runs in survive after the task ends, or should it evaporate? An ephemeral sandbox is a fresh machine per task — the agent does its work, you read the result, and the whole thing is destroyed, taking every trace with it. A stateful sandbox is the opposite bet: the machine, or a snapshot of its state, sticks around between turns and sessions, so the agent keeps its filesystem, its installed packages, its warm caches, and whatever it was in the middle of doing. Both are correct designs. Picking the wrong one for your workload costs you either latency and cache misses, or money and a data-leak footgun. This is the post about which one to pick.

I'm Ajay — I build PandaStack, an open-source Firecracker microVM platform, so I spend most of my time thinking about exactly what happens to a machine after an agent is done with it. The short version: default to ephemeral, reach for stateful only when the agent genuinely needs continuity, and never, ever reuse one stateful sandbox across two tenants. The long version is below.

What "state" even means for an agent

"Stateful" gets thrown around loosely, so it's worth being precise about what actually accumulates inside a running agent sandbox. When people say an agent has state, they usually mean some mix of these, and they don't all live in the same place:

• Filesystem — the files the agent wrote: a cloned repo, generated code, downloaded datasets, scratch output from a previous step. This is the most obvious kind of state and the one people picture first.
• Installed packages and toolchains — `pip install`, `npm install`, `apt-get`, a compiled binary, a `mise install` of a specific Node version. Rebuilding these on every task is often the single biggest latency tax an agent pays.
• Warm caches — a populated package cache, a built dependency tree, a warmed model or embedding cache, a database with its buffer pool full. Nothing here is semantically important, but re-warming it is slow.
• In-memory working context — a long-running process the agent started, a REPL session, variables in a notebook kernel, a partially-built in-memory index. This lives in RAM and is the hardest kind of state to reconstruct.
• Open connections — an authenticated database session, an SSH tunnel, a websocket to an upstream service, a logged-in CLI. These are stateful and they're also the ones most likely to hold credentials.
• Browser cookies and auth — for browser-using agents, the logged-in session, cookies, local storage, and saved credentials that let the agent stay signed in across turns instead of re-authenticating every time.

The reason this list matters is that ephemeral throws away all six, every single time, and stateful keeps all six, every single time. That's the whole tradeoff in one sentence — but each of those six is also a thing that can leak, cost money, or go stale, which is why the decision isn't as simple as "keep more state, get more speed."

Ephemeral: fresh machine, total amnesia

An ephemeral sandbox is created for one task and destroyed when it's done. The agent gets a clean, known-good machine every time — same starting filesystem, same packages, no residue from whatever ran last. When the task finishes (or blows a timeout, or the orchestration process crashes), the VM is reaped and everything it touched vanishes with it. This is the strongest posture on two axes at once, and both matter.

First, isolation. Nothing survives between runs, so there is no residual state for one task to leak into the next. A secret the previous task wrote to `/tmp`, a dataframe still resident in memory, a logged-in session in a cookie jar — none of it can reach the next task, because there is no next task on that machine. Second, cost. An idle ephemeral sandbox doesn't exist, so it costs nothing. If a tenant runs one agent task and then goes quiet for three days, you're paying for zero machines during those three days. For bursty, one-shot, or scheduled agent work — which is most agent work — that's exactly the shape you want.

The cost of ephemeral is the flip side of its virtue: amnesia. If the agent needs the repo it cloned two turns ago, it clones it again. If it `pip install`ed a heavy dependency, it installs it again. If it had a notebook kernel with an hour of computed state, that state is gone. For a lot of workloads that re-work is cheap and the clean slate is worth it. For some, it's a dealbreaker — and that's the case for stateful.

Stateful: continuity across turns

A stateful (persistent) sandbox survives past the end of a task. The same machine — or a snapshot of its exact state — is there for the next turn, the next tool call, the next session. The agent picks up where it left off: the repo is still cloned, the dependencies are still installed, the caches are still warm, the notebook kernel still has its variables, the browser is still logged in. For an interactive session where the agent and a human are iterating together over minutes or hours, this is the difference between a responsive tool and one that spends the first ten seconds of every turn rebuilding the world.

Stateful is the right call when continuity is the product: a coding agent working through a long multi-step task in one repo, a hosted notebook a user pokes at all day, a browser agent that logs in once and then performs a sequence of actions, a data agent that loads a big dataset once and then answers many questions against it. In all of these, the state isn't incidental — it's the thing that makes the next turn fast and coherent. Throwing it away every turn would make the agent slower and dumber.

But stateful buys you two liabilities in exchange for that continuity, and they're the whole reason this decision deserves a blog post: residual state can leak, and idle machines cost money. Both are manageable. Neither is optional to think about.

The security tradeoff: residual state is a leak surface

Here's the load-bearing security fact. Ephemeral sandboxes are self-cleaning: because nothing survives, there's no residual data for one use to leak into the next. Stateful sandboxes are the opposite by design — they exist precisely to carry data forward. That's fine, and even the point, as long as the thing the data is carried forward to is the same trust boundary it came from. The catastrophe is reusing a stateful sandbox across trust boundaries.

Concretely: a persistent VM that ran tenant A's agent has tenant A's files on disk, tenant A's data in memory, tenant A's credentials in a logged-in session, tenant A's cookies in the browser. If you then hand that same persistent VM to tenant B to save a create, you have just built the shared-worker-pool data leak by hand — tenant B's agent is now running on a machine full of tenant A's residue, and a curious or hostile prompt can read all of it. The VM boundary that was supposed to protect tenants from each other is gone, because you reused the machine across the boundary it was supposed to enforce.

Note the asymmetry: ephemeral makes this mistake structurally impossible, because there's no VM to reuse in the first place. That's a real part of ephemeral's appeal — it's not just cheaper, it's harder to hold wrong. Stateful gives you continuity but hands you the responsibility of never crossing tenants with it. If your workload is multi-tenant and you're not confident you can guarantee one-tenant-per-VM discipline, that's a strong signal to default ephemeral and only go stateful per-tenant, deliberately. There's a whole post on the per-tenant boundary at /blog/microvm-per-tenant-analytics-query if you want the deeper version of this argument.

The cost tradeoff: idle stateful VMs cost money

The second liability is money. An ephemeral sandbox that isn't running costs nothing because it doesn't exist. A persistent sandbox, by definition, exists whether or not anyone is using it — and a machine that exists consumes host memory and CPU budget even while its agent sits idle waiting for the next turn. Keep a persistent VM per tenant and let a thousand tenants go idle overnight, and you're paying for a thousand idle machines to preserve state nobody is touching.

The escape hatch is to persist the state without keeping the machine running. Snapshot or hibernate captures the full machine — filesystem and memory — into storage and then stops the VM, so the state survives but the idle compute cost trends toward zero. On the next request you restore from that snapshot instead of holding a live VM the whole time. This is the move that makes stateful economical: you get warm-resume continuity without paying for a warm machine 24/7. There's a dedicated post on this at /blog/long-running-sandboxes-for-ai-agents, and one specifically on preserving an agent's working memory at /blog/ai-agent-persistent-memory-state.

Ephemeral vs. stateful, side by side

• Isolation — Ephemeral: strongest; nothing survives a run, so cross-run leaks are structurally impossible and there's no VM to reuse by mistake. Stateful/persistent: state is carried forward by design, so residual data exists and MUST stay within one trust boundary.
• Idle cost — Ephemeral: zero; an idle sandbox doesn't exist, so idle tenants cost nothing. Stateful/persistent: a live persistent VM costs money while idle unless you hibernate/snapshot it to storage and restore on demand.
• State survival — Ephemeral: none; filesystem, deps, caches, in-memory context, and sessions are all gone at teardown. Stateful/persistent: everything survives — files, installed packages, warm caches, notebook kernels, open connections, browser auth.
• Cross-tenant safety — Ephemeral: safe by construction; there's no reused machine to leak across tenants. Stateful/persistent: safe ONLY as one-tenant-per-VM; reuse across tenants rebuilds the shared-pool data leak.
• Best fit — Ephemeral: one-shot tasks, scheduled jobs, API-driven runs, untrusted or model-generated code, anything bursty. Stateful/persistent: interactive sessions, long multi-step tasks in one workspace, hosted notebooks, browser agents that stay logged in, big-dataset agents.

The two shapes in code

On PandaStack the difference is a couple of fields, because every create is a snapshot-restore either way — the restore step lands around 49ms, with an end-to-end create at p50 179ms and p99 about 203ms; only the very first spawn of a template pays the ~3s cold boot. So an ephemeral sandbox isn't expensive to create; it's just configured to die on its own. Here's ephemeral: a `with` block plus a `ttl_seconds` backstop, and the whole VM is gone at the end of the block.

from pandastack import Sandbox

# EPHEMERAL: fresh microVM for one task, dies at the end of the block.
# Nothing survives -> strongest isolation, zero idle cost, total amnesia.
def run_one_shot(user_task: str) -> str:
    with Sandbox.create(
        template="code-interpreter",
        ttl_seconds=300,          # backstop: a leaked VM reaps ITSELF
    ) as sbx:
        sbx.filesystem.write("/workspace/task.py", user_task)
        result = sbx.exec("python3 /workspace/task.py", timeout_seconds=60)
        return result.stdout
    # VM destroyed here. Files, deps, memory, sessions -> all gone.
    # The next task starts from a clean, known-good machine.

And here's stateful: create the sandbox with `persistent=True` so the idle reaper leaves it alone, keep its id, and reuse it across turns. The critical bookkeeping is that this sandbox belongs to exactly one tenant for its entire life, and you destroy it explicitly when the session (or the account) ends — nothing tears it down for you.

from pandastack import Sandbox

# STATEFUL: one persistent microVM per tenant, reused across turns.
# persistent=True exempts it from the idle reaper and pins it to its host.
# HARD RULE: this VM serves ONE tenant, forever. Never hand it to another.
def get_or_create_session(tenant_id: str, saved_id: str | None) -> Sandbox:
    if saved_id:
        return Sandbox.get(saved_id)          # resume: repo, deps, caches intact
    sbx = Sandbox.create(
        template="code-interpreter",
        persistent=True,                      # survives past the task
        metadata={"tenant_id": tenant_id},    # so the boundary is auditable
    )
    sbx.exec("git clone https://example.com/acme/repo /workspace && "
             "cd /workspace && pip install -r requirements.txt")
    return sbx                                # store sbx.id against tenant_id

# Turn 2, turn 3, turn N -> reuse the SAME sandbox. The clone and the
# pip install from turn 1 are still there; no re-work, warm and fast.

# When the session (or account) ends, tear it down yourself -- nothing else will:
def end_session(sbx: Sandbox) -> None:
    sbx.destroy()                             # persistent VMs don't self-reap

Two SDK details do the load-bearing work. `persistent=True` is what exempts the VM from the idle reaper and pins it to its host — without it, an ephemeral VM's TTL would eventually reap the session out from under your user. And storing `sbx.id` keyed by `tenant_id` is both how you resume and how you enforce one-tenant-per-VM: the lookup that finds the sandbox is the same lookup that guarantees you never hand tenant B a machine you created for tenant A.

A decision framework

Here's how I'd actually decide, in order. Start from ephemeral and only move toward stateful when a concrete need forces you.

1. Does anything need to survive between runs? If no — the task is self-contained, produces a result, and the next task starts fresh — use ephemeral and stop here. This is most one-shot tasks, scheduled jobs, and API-driven runs, and it's the default for a reason.
2. Is the code untrusted or model-generated? Lean ephemeral even if a little state would be convenient. A machine that's destroyed after every run can't accumulate a foothold, and the clean-slate-every-time property is worth more than the caching you'd save.
3. Is the agent in an interactive, multi-turn session where warmth is the experience? This is the real case for stateful — a human iterating with the agent, a hosted notebook, a browser agent staying logged in. Re-cloning and re-installing on every turn would make it slow and frustrating. Go stateful.
4. If stateful, is it strictly one tenant per persistent VM? If you can't guarantee that, don't go stateful for that workload — the cross-tenant leak isn't worth the caching. If you can, tag the VM with the tenant and keep the boundary auditable.
5. If stateful, will the VM sit idle between bursts? Then don't hold a live VM — hibernate/snapshot it to storage and restore on the next request, so you keep the state without paying for idle compute. Reserve always-on live VMs for sessions where even a fast restore is too slow.
6. Still unsure? Go ephemeral. It's the safer, cheaper default, and the one that's structurally hard to hold wrong. You can always promote a specific workload to stateful later; walking back an accidental cross-tenant leak is a much worse afternoon.

The pattern most mature platforms land on is a hybrid: ephemeral for the long tail of one-shot and scheduled agent tasks, and a stateful per-tenant VM only for the interactive feature where continuity is the product — hibernated between bursts so idle cost trends to zero and a returning user gets a warm resume. You don't have to pick one globally. You pick per workload, and the default is ephemeral.

The honest bottom line

Ephemeral and stateful are two answers to one question — should the machine survive the task — and they trade the same two things in opposite directions. Ephemeral gives you the strongest isolation (nothing survives, so nothing leaks, and there's no VM to reuse across tenants by mistake) and zero idle cost (an idle sandbox doesn't exist), at the price of amnesia between runs. Stateful gives you continuity — files, deps, warm caches, in-memory context, browser auth all carried forward — at the price of residual state you must keep inside one trust boundary and idle compute you must hibernate away. Default to ephemeral; it's cheaper, safer, and harder to hold wrong. Reach for stateful when an interactive or long-running session genuinely needs the warmth — and when you do, hold the line on one tenant per persistent VM, forever, and snapshot the idle ones so state costs you memory, not money. Get that right and the choice stops being a tax and starts being a tool.