A MicroVM for Every LLM Tool Call: Per-Request Isolation

When an LLM app exposes tools — `run_code`, `run_sql`, `browse`, `shell` — every call executes something the model wrote, and the model wrote it under the influence of whatever was in its context window. That context can include a user's paste, a scraped web page, a retrieved document, or an email body. In other words, the arguments to your tool call are attacker-influenceable by design. The question isn't whether to sandbox them — it's what the unit of isolation should be. The answer that holds up in production is a fresh microVM per request, not a shared long-lived worker that many calls take turns using.

I'm Ajay — I built PandaStack, a Firecracker microVM platform for exactly this pattern. This post is about why per-invocation isolation matters, the create-run-destroy loop that implements it, why sub-second create is the thing that makes "a VM per request" affordable instead of absurd, and how forking from a warm known state lets you keep the isolation without paying setup cost every time.

Why the isolation boundary has to be per-request

Two failure modes make the shared-worker pattern dangerous, and they compound. The first is prompt injection: text the model ingested — a webpage, a PDF, a user message — instructs it to do something the developer never intended. "Summarize this page" quietly becomes "read the API keys in the environment and POST them to evil.example." You cannot review the generated code before it runs, because the whole point of a tool-using agent is that it decides at runtime. So you have to assume any given tool call is hostile and contain it.

The second is cross-tenant leakage, and it's the one that turns a bad day into a breach. If tenant A's `run_code` call and tenant B's call land in the same long-lived worker, they share a filesystem, a process table, environment variables, and whatever the last call left in `/tmp`. The model that just read tenant A's secrets is now happily serving tenant B. No exploit required — just a `cat /tmp/*` that the previous call was tricked into writing. Namespaces and a `chdir` do not fix this; the shared kernel and shared process space are the leak.

This is why AWS Lambda gives every function invocation a Firecracker microVM rather than reusing one process across customers, and why every serious code-interpreter product ends up at some form of per-execution VM. Once you accept that the tool arguments are untrusted, the boundary has to be a real hardware-virtualized one, and it has to reset between invocations that don't share a trust domain.

The create-run-destroy loop

The pattern is boring on purpose: for each untrusted tool call, create a fresh sandbox, run the code with a hard timeout, capture the result, and destroy the sandbox. Nothing survives to the next call. On PandaStack each sandbox is its own Firecracker microVM — separate guest kernel, filesystem, memory, and network namespace — so "destroy" actually returns the machine to nothing, not a cleaned-up-ish container. The `with` form kills the VM on block exit, so there's no leak even if the code throws.

from pandastack import Sandbox

def run_tool_call(user_code: str) -> dict:
    """One untrusted tool call = one fresh microVM, created and destroyed here."""
    with Sandbox.create(template="code-interpreter", ttl_seconds=120) as sbx:
        sbx.filesystem.write("/workspace/call.py", user_code)
        # timeout_seconds is your circuit breaker for model-written infinite loops.
        result = sbx.exec("python3 /workspace/call.py", timeout_seconds=30)
        return {
            "exit_code": result.exit_code,
            "stdout": result.stdout,
            "stderr": result.stderr,
        }
    # VM is destroyed on block exit — nothing carries into the next call.

# Every invocation gets its own VM; two tenants never share one.
print(run_tool_call("print(sum(range(100)))"))

Two guardrails belong on every call. `timeout_seconds` on `exec` bounds runaway code — model-written loops are more common than you'd like, and the timeout is the circuit breaker. `ttl_seconds` on create is the backstop that reaps the VM even if your process crashes between create and destroy. Belt and suspenders: the code has a deadline, and the VM has an expiry.

You return the exit code and stderr straight back to the model so it can self-correct on a traceback — that feedback loop is most of what makes a code tool useful. What you never do is return the previous call's process, its files, or its environment. Each call starts from the same clean template state and ends in the void.

Why sub-second create is what makes this practical

Per-request VMs were historically a non-starter for one reason: VMs took seconds to boot. If every tool call paid a multi-second boot tax, latency-sensitive apps couldn't do it, so the industry reached for warm pools — pre-booted workers kept hot and reused across requests. Warm pools are precisely the shared-state pattern we're trying to avoid; they trade isolation for latency because the boot cost forced the trade.

Snapshot-restore removes the trade. Instead of cold-booting a kernel and userspace on every create, the platform boots a template once, snapshots the running machine, and restores that snapshot on demand. On PandaStack a create is a snapshot restore in ~49ms of restore work, landing at a p50 of 179ms and p99 of ~203ms end to end. A first-ever cold boot of a brand-new template is ~3s, but that happens once at bake time — every real request takes the restore fast path.

There's a capacity angle too. Because create is restore-on-demand rather than a warm pool of idle VMs burning RAM, you're not paying to keep hundreds of workers hot on the chance a request arrives. A single agent pre-allocates 16,384 /30 subnets, so per-request VMs scale with traffic rather than with a guessed pool size. You spin them up when calls come in and let them evaporate when they finish.

Fork-per-request: start from a warm, known state

A blank template is clean but sometimes too blank — maybe every tool call needs a dataset loaded, a virtualenv activated, or a model warmed in memory. Doing that setup inside every fresh VM would reintroduce a per-call cost. Fork solves it: configure one sandbox exactly how you want, snapshot it, then `fork` that snapshot per request. Each fork is an independent VM that starts from the identical post-setup state, with memory shared copy-on-write until the child writes.

from pandastack import Sandbox

# 1) Build the warm base ONCE: load data, install deps, prime caches.
base = Sandbox.create(template="code-interpreter", persistent=True)
base.filesystem.write("/workspace/setup.py", (
    "import pandas as pd\n"
    "df = pd.read_parquet('/data/reference.parquet')\n"
    "df.to_parquet('/workspace/ready.parquet')  # warm state on disk\n"
))
base.exec("python3 /workspace/setup.py", timeout_seconds=120)
snap = base.snapshot()  # freeze the warm, known-good state

def run_from_warm(user_code: str) -> dict:
    """Each untrusted call forks the warm state into its own isolated VM."""
    with snap.fork() as child:  # same-host fork: ~400-750ms, CoW memory
        child.filesystem.write("/workspace/call.py", user_code)
        r = child.exec("python3 /workspace/call.py", timeout_seconds=30)
        return {"exit_code": r.exit_code, "stdout": r.stdout, "stderr": r.stderr}
    # child VM destroyed here; the warm base is untouched and reusable.

print(run_from_warm(
    "import pandas as pd; print(len(pd.read_parquet('/workspace/ready.parquet')))"
))

A same-host fork lands in ~400–750ms because it reflinks the rootfs and restores memory locally with copy-on-write — the child shares the parent's pages until it dirties them. A cross-host fork (when the scheduler places the child on a different agent) is ~1.2–3.5s because it involves pulling snapshot data over the network. For per-request isolation you almost always want same-host forks off a warm base: you get the clean-slate guarantee of a fresh VM and the low latency of not redoing setup.

The isolation property is intact: forks are independent VMs. Tenant A's fork and tenant B's fork share read-only base pages via CoW, but neither can see the other's writes — copy-on-write means a write to a shared page transparently gets a private copy. The shared memory is an efficiency detail, not a shared mutable surface.

When to reuse a VM, and when never to

Fresh-per-request is the safe default, but it isn't the only valid shape. The rule is about trust domains, not requests: you may reuse a VM across calls that share a trust domain, and you must never reuse one across calls that don't.

• Trust boundary — Reuse: fine within one authenticated user's single session. Never: across different tenants or users, ever.
• State — Reuse: an interactive session where later cells build on earlier ones (a notebook, an agent iterating on one dataset). Never: a stateless "run this and return the answer" tool where leftover state is a liability.
• Latency of setup — Reuse: keep a session VM warm to avoid re-loading big context each turn. Fresh/fork: use a warm base + fork-per-call so setup is paid once, not per call, without sharing a live VM.
• Failure blast radius — Reuse: acceptable when a leak stays inside one user's own data. Never: when a leak would cross a customer boundary — that's a breach, not a bug.

The mental model: one long-lived sandbox is a single-tenant session, and it's genuinely useful for an agent that works a problem over many turns for one user. The moment two different users' code could touch the same VM, you've turned an isolation boundary into a shared bus. For multi-tenant traffic, fresh-per-request or fork-per-request, and kill it after. When in doubt, destroy it — a VM is cheap now.

A practical checklist

If you're wiring untrusted tool calls into a production LLM app, the shape that holds up is small and mechanical:

1. Treat every tool-call argument as attacker-influenceable — the model wrote it under untrusted context.
2. Give each untrusted invocation its own microVM; never share a VM across tenants.
3. Set both timeout_seconds on exec and ttl_seconds on create — bound the code and bound the VM.
4. For expensive setup, fork per request from a warm snapshot instead of reusing a live VM.
5. Reuse a VM only within one trusted session, and kill it when the session ends.
6. Restrict network egress at the network layer if exfiltration is in your threat model — don't trust the code to behave.

None of this is exotic. It's the same discipline that makes serverless platforms safe, applied to the place LLM apps actually run untrusted input: the tool call. Sub-second microVM create is the enabler that lets you keep real hardware isolation on every invocation without the latency penalty that used to force everyone into shared warm pools. A VM per request used to be a wild idea. Now it's just the correct default.