Robotics Simulation and RL Rollouts in Isolated microVMs

Robotics simulation has a peculiar operational shape. The compute is embarrassingly parallel — a thousand rollouts of one policy in one world are a thousand independent jobs with nothing to say to each other. But each job is pathologically sensitive to its environment: a different ROS 2 patch release, a different libstdc++, a different BLAS threading default, and the trajectory diverges. And increasingly the controller being evaluated wasn't written by anyone you can call — it came from an AI agent, a student submission, or a competitor in an eval harness. So you need N clean, identical, mutually-invisible Linux machines on demand, any of which might be handed hostile code. That's a microVM-shaped problem wearing a robotics hat.

I'm Ajay, I built PandaStack, so treat this as an opinionated tour rather than a neutral survey. I'll be specific about what the microVM model buys a sim pipeline — a pinned environment, per-rollout network islands, cheap fan-out from a warmed snapshot, safe execution of untrusted controllers — and equally specific about where it doesn't help. If your bottleneck is photoreal rendering on a datacenter GPU, this post won't save you, and I'd rather say that in paragraph two than in the conclusion.

Why simulation is a microVM-shaped workload

Most sandboxed workloads are request-shaped: short, stateless, latency-sensitive. Simulation is none of those. MicroVMs still fit because four properties of the workload line up almost suspiciously well with four properties of the runtime.

• Embarrassingly parallel → per-VM isolation costs nothing in coordination. Episodes share no state, so there's no cross-VM chatter to make expensive — you're paying for N independent Linuxes, not a distributed system.
• Environment-sensitive → a VM image pins more than a container image. The guest kernel is part of the artifact, not inherited from whatever host you landed on.
• Chatty middleware → a private network namespace is a hard mute button. ROS 2 nodes discover each other automatically; a per-sandbox netns means each rollout finds only its own.
• Arbitrary submitted code → hardware virtualization is the boundary you actually want. A controller that shells out and deletes things deletes them inside a throwaway guest, not on a shared host kernel.

Reproducibility: the snapshot IS the environment

Why "pin the image" isn't enough

"Works on my ROS install" is the robotics variant of the oldest joke in software, and it's worse than the general case because the dependency graph is enormous and half of it is native code: DDS middleware, a physics engine, mesh and collision libraries, image codecs, a linear algebra stack, and a Python layer on top — mostly compiled, much of it version-pinned only by convention. Two machines with the same apt manifest can still produce different trajectories because one has a different kernel, a different glibc, or a CPU that takes a different path inside a hand-tuned BLAS routine.

A container image pins user space — a real improvement, and why everyone containerized their sim stack years ago. But a container borrows the host's kernel, so the bottom of your stack (scheduling, timers, futexes, allocator behavior under transparent hugepages) is whatever the host runs today. Upgrade a node pool and your "pinned" environment silently changed underneath you. A microVM snapshot includes the guest kernel: bake a template, restore it, and you get a specific kernel, init state and user space as one artifact.

The apparatus, versioned

It goes further than "the same files." On PandaStack every sandbox create restores a baked snapshot — no warm pool of long-lived machines drifting apart over a week of use. Rollout one and rollout nine thousand start from bit-identical memory and disk state: a stronger guarantee than "we ran the same image," and the difference between a result you can defend in a paper and one you can only defend in a standup.

The ROS 2 discovery problem: your nodes are too friendly

Here is the failure mode that bites every lab scaling sim horizontally for the first time. ROS 2 uses DDS, and DDS is built around automatic peer discovery — nodes announce themselves and find each other over multicast, with no central broker. Lovely design for a robot, where the perception node and the control node should just find each other. Catastrophic design for a shared cluster, where you've started forty copies of the same experiment on machines that can all see each other.

What happens is that rollout 7's /cmd_vel publisher discovers rollout 23's /cmd_vel subscriber, and two experiments start silently steering each other's robots. Nobody gets an error. The topics match, the QoS profiles are compatible, the data flows. You just get results that are subtly, unreproducibly wrong. The standard mitigations — a distinct ROS_DOMAIN_ID per run, restricted discovery ranges, binding the DDS vendor to one interface — all work, and all are conventions your launcher must get right every single time; the domain ID space is also small enough that at real fan-out you start recycling it.

A per-sandbox network namespace removes the ambient authority entirely. Each PandaStack sandbox gets its own netns, tap device and /30 subnet (an agent pre-allocates 16,384, so per-rollout networking isn't the ceiling). Rollout 7's multicast packets have nowhere to go but rollout 7's own interfaces. Still set the domain ID and DDS config — defense in depth — but a convention is no longer the only barrier between independent experiments.

Determinism, and the clone-RNG trap

Isolation gets you a clean environment; it does not get you determinism. That's its own discipline — fixed timestep, fixed solver iterations, pinned thread counts so floating-point reductions don't reorder, no wall-clock-dependent logic, every RNG seeded. Mostly simulator config, not infrastructure. But there is one trap infrastructure introduces, specific to the snapshot-and-fork pattern this post recommends.

This is the funniest possible bug because the infrastructure is working perfectly — you asked for exact clones and got exact clones. It also doesn't announce itself: variance across your "parallel" runs quietly collapses to zero, confidence intervals get suspiciously tight, and the policy looks great until it meets a world it hasn't memorised. The fix is mundane. Pass the seed into each guest after restore and derive every source of randomness from it, including the ones you forgot about — language-level hash seeds and per-library generators both count.

Here's the entrypoint that runs inside each rollout's guest. It pins the knobs a snapshot can't capture, clamps DDS to the local machine, and refuses to start without an explicit seed.

#!/usr/bin/env bash
# /opt/sim/rollout.sh -- runs INSIDE one rollout's microVM.
# The template snapshot already froze the kernel, the ROS 2 distro and every
# .so. This script pins the runtime knobs a snapshot can't capture.
set -euo pipefail

source "/opt/ros/${ROS_DISTRO:-jazzy}/setup.bash"

# --- 1. Mute DDS. The sandbox already has its own network namespace, so
#        there is no shared L2 segment to discover peers on -- but say it
#        out loud anyway so the intent survives the next refactor.
export ROS_DOMAIN_ID=0
export ROS_AUTOMATIC_DISCOVERY_RANGE=LOCALHOST
export RMW_IMPLEMENTATION=rmw_cyclonedds_cpp
export CYCLONEDDS_URI='<CycloneDDS><Domain><General><Interfaces>'\
'<NetworkInterface name="lo"/></Interfaces>'\
'<AllowMulticast>false</AllowMulticast></General></Domain></CycloneDDS>'

# --- 2. Pin everything that makes floating point reorder itself.
export OMP_NUM_THREADS=1
export MKL_NUM_THREADS=1
export OPENBLAS_NUM_THREADS=1
export MUJOCO_GL=osmesa      # headless software GL: no X, no GPU, no display

# --- 3. Seed EXPLICITLY. A forked VM inherits its parent's RNG state, so
#        "just use the default seed" means N identical episodes. Fail loud.
SEED="${EPISODE_SEED:?EPISODE_SEED must be set by the caller}"
export PYTHONHASHSEED="$SEED"

exec python3 -u /opt/sim/run_episode.py \
  --world      /opt/sim/worlds/warehouse.sdf \
  --controller /workspace/controller.py \
  --seed       "$SEED" \
  --timestep   0.002 \
  --solver-iterations 50 \
  --max-steps  20000 \
  --out        "/data/episodes/${SEED}.jsonl"

Fan-out: warm the sim once, fork it per episode

The expensive part of a rollout is often not the rollout. Loading a warehouse world, convex-decomposing meshes, building collision structures, JIT-warming the physics step, importing a Python stack the size of a small city — that's all paid before step one, and a naive fan-out pays it N times. For short episodes it dominates: you can spend more wall clock on world loading than on the physics you wanted to measure.

Snapshot-and-fork collapses that to once. Boot one sandbox, warm it until the simulator sits at step zero with everything parsed and resident, snapshot it, then fork per episode. Each child wakes already warm — the meshes are parsed because the parent parsed them, and the memory holding them is shared copy-on-write until a child writes. A same-host fork lands in 400–750ms; cross-host is 1.2–3.5s because the memory has to travel.

Copy-on-write is also what makes the density tolerable: five hundred forks don't cost five hundred times one sim's RAM, because the pages nobody writes — kernel, engine code, immutable world geometry — are shared. Divergence costs memory; identity is free.

"""Fan out N RL episodes from one warmed simulator snapshot."""
from concurrent.futures import ThreadPoolExecutor, as_completed

from pandastack import Sandbox

WORLD = "warehouse"
EPISODES = 256


def run_episode(parent: Sandbox, controller: bytes, seed: int) -> dict:
    """One episode in its own microVM, forked from the warmed parent."""
    # Same-host fork: 400-750ms, and the world is ALREADY parsed because the
    # parent parsed it. Memory is copy-on-write until this child diverges.
    with parent.fork() as child:
        # Untrusted controller code -- an agent wrote this. It gets its own
        # guest kernel, its own disk, and its own network namespace.
        child.filesystem.write("/workspace/controller.py", controller)

        # THE IMPORTANT LINE: the fork inherited the parent's RNG state, so
        # every child would replay the same episode unless we reseed here.
        res = child.exec(
            f"EPISODE_SEED={seed} bash /opt/sim/rollout.sh",
            timeout_seconds=1800,
        )
        if res.exit_code != 0:
            return {"seed": seed, "ok": False, "error": res.stderr[-2000:]}

        summary = child.exec(
            f"tail -n 1 /data/episodes/{seed}.jsonl", timeout_seconds=30
        )
        return {"seed": seed, "ok": True, "result": summary.stdout.strip()}
    # Child destroyed on block exit. Whatever the controller did dies with it.


def evaluate(controller: bytes, base_seed: int = 1_000) -> list[dict]:
    # 1. Boot ONE sandbox and pay the warm-up cost exactly once.
    with Sandbox.create(
        template="base",
        ttl_seconds=7200,
        metadata={"role": "sim-parent", "world": WORLD},
    ) as parent:
        warm = parent.exec(
            f"bash /opt/sim/warmup.sh --world {WORLD}", timeout_seconds=900
        )
        assert warm.exit_code == 0, warm.stderr

        # 2. Freeze the warmed simulator: world loaded, meshes parsed,
        #    physics initialised, sitting at step zero.
        parent.snapshot()

        # 3. Fork it once per episode, each with its own explicit seed.
        with ThreadPoolExecutor(max_workers=32) as pool:
            futures = [
                pool.submit(run_episode, parent, controller, base_seed + i)
                for i in range(EPISODES)
            ]
            return [f.result() for f in as_completed(futures)]


if __name__ == "__main__":
    results = evaluate(open("policy_controller.py", "rb").read())
    ok = [r for r in results if r["ok"]]
    print(f"{len(ok)}/{len(results)} episodes completed")

The SDK surface is deliberately boring: create, filesystem.write, exec with a timeout, snapshot, fork. The robotics-specific thinking is entirely in what you choose to snapshot — freeze at the most expensive reusable moment, which for simulation is "everything loaded, nothing simulated yet."

Untrusted controller code: agents, students, competitors

Sim harnesses used to assume the controller came from a colleague. That assumption is dead. If a language model proposes the code, or you run an autograder or a public leaderboard, arbitrary code executes with whatever privileges your harness has — and harnesses are unusually generous: mounted datasets, credentials for metric logging, a network shared with the rest of the cluster.

A container here is, as the saying goes, a polite suggestion to the kernel. Namespaces and cgroups are real controls, but every container on the box is one shared kernel deep, and you're inviting model-authored code — which will cheerfully `rm -rf` a path it hallucinated, or notice that reading the ground-truth trajectory file scores better than actually controlling the robot — to poke at that kernel's full syscall surface. A microVM makes the boundary hardware-enforced: own guest kernel, own disk, own netns, escape requires breaking the hypervisor rather than finding one kernel bug. And because the episode VM is discarded at the end of the block, reward hacking that writes to the environment can't leak into the next episode.

Long-running jobs and hibernating between batches

Persistent sandboxes for multi-day runs

Not all sim work is short. Curriculum training runs for days; a single high-fidelity episode can take an hour of CPU. Ephemeral sandboxes with a TTL cover the short shape; for the long shape use a persistent sandbox with a durable volume, so checkpoints and trajectory logs live on real disk rather than an ephemeral rootfs and the machine is exempt from the idle reaper.

Hibernate between experiment batches

Hibernation between batches saves the most money. Research doesn't run continuously — you launch a sweep, it finishes, you stare at plots for a day, then you launch the next against the same warmed environment. A hibernated sandbox is snapshotted to storage and stopped, so an idle experiment costs storage rather than compute, and waking it restores the exact state you left, loaded world included. Scale-to-zero for research infrastructure: stop paying for a cluster that spends most of its week waiting for a human to read a graph.

Workstation vs container vs microVM vs full VM

Four ways to run a sim fleet. Verify the specifics of any runtime, DDS vendor or hypervisor against its own docs — behavior varies a lot by version and configuration, especially around discovery defaults and GPU support.

• Reproducibility — Workstation: whatever you last apt-upgraded; irreproducible by construction. Container: user space pinned, host kernel borrowed and free to change under you. microVM: kernel, user space and memory state pinned in one snapshot. Full VM: same pinning, but boots are slow enough that people reuse long-lived VMs and let them drift.
• Network isolation for DDS — Workstation: one LAN, automatic discovery, silent cross-talk. Container: shared bridge by default; workable with per-run networks and domain IDs, but that's convention you enforce every time. microVM: per-sandbox netns and tap device — each rollout is its own island with no shared L2. Full VM: same isolation at much higher per-instance cost.
• Parallel fan-out cost — Workstation: bounded by the one box under your desk. Container: cheap to start, but each one re-parses the world and re-warms the physics. microVM: snapshot a warmed sim and fork it — same-host fork 400-750ms, copy-on-write memory, warm-up paid once. Full VM: minutes to provision; nobody forks per episode.
• GPU access — Workstation: full, direct, easy — this is its whole argument. Container: mature GPU passthrough via vendor runtimes; the standard choice for GPU sim. microVM: not the Firecracker story — assume CPU physics only. Full VM: possible with device passthrough, but real work, and it pins you to specific hosts.
• Untrusted code safety — Workstation: none; the submission runs as you. Container: namespaces and cgroups over a shared host kernel — one kernel bug from a cross-tenant problem. microVM: hardware-virtualized guest kernel per rollout; escape needs a hypervisor break. Full VM: equally strong, but too slow per submission, so people batch into one VM and lose the isolation they paid for.
• Time to a fresh clean environment — Workstation: manual, minutes to hours, occasionally a reinstall. Container: seconds, plus image pull on a cold node. microVM: p50 179ms, p99 ~203ms via snapshot-restore (~3s only for the first-ever cold boot, before a snapshot exists). Full VM: minutes.

Where microVMs are not the answer

Blunt about the ceiling: GPU passthrough is not the Firecracker story. Firecracker deliberately exposes a minimal virtio device model — that tiny attack surface is exactly why it's safe to hand untrusted code — and "minimal device model" and "pass through a datacenter GPU" are close to opposites. If your workload is photoreal rendering, a vision model training on synthetic imagery, or a physics engine whose whole pitch is thousands of GPU-parallel environments, a microVM fleet is the wrong tool and fast snapshot-restore doesn't fix it. Same for hardware-in-the-loop: a rig needing a real USB device, a CAN interface, or a real-time kernel driving actual motors wants a machine wired to that hardware, not a guest with an emulated NIC.

What's left is still most of the work: CPU-side rigid-body and contact physics, headless ROS 2 integration tests, policy-evaluation rollouts where the environment rather than the policy is the cost, domain-randomization sweeps, regression suites over recorded scenarios, and any harness executing controller code somebody else wrote. For that set the microVM model gives you a genuinely pinned environment instead of an aspirationally pinned one, network islands that make DDS cross-talk structurally impossible rather than merely discouraged, fan-out that pays world-loading once, and a hardware boundary around code you have no reason to trust. Just remember to reseed after the fork — otherwise you get five hundred beautifully isolated, perfectly reproducible copies of the same episode, the most elegant way I know to waste a compute budget.