Running User-Generated Game Mods in Isolated microVMs

Every game with a mod API has shipped a remote code execution feature and called it a community platform. That's not a criticism — user-generated content is often the entire reason a game outlives its launch window. Minecraft's plugin ecosystem, Roblox's creator economy, Steam Workshop, Factorio mods, community-run shards and private servers: the modders are the product. But the operational reality underneath is uncomfortable. Somebody you have never met writes a script, uploads it, and your infrastructure runs it. In-process. Next to other players' session data. On a kernel you also use for everything else.

The standard answer has been a language-level sandbox: a locked-down Lua state, a JS VM with the dangerous globals deleted, a QuickJS interpreter with a restricted host API. I'm Ajay, I built PandaStack, and this post argues that those sandboxes are a usability feature that got mistaken for a security control — and that the actual boundary you want is a Firecracker microVM per mod session or per shard. Not because language sandboxes are useless, but because their track record against motivated attackers is genuinely bad, and the historical reason you couldn't just give each mod a VM (VMs are slow and heavy) stopped being true.

What a hostile mod actually does

Threat-modeling mods is unusually easy because the attackers publish their work. You do not have to imagine what someone would do with code execution on a game server. Four patterns cover almost everything.

Mining, proxying, and other free-compute theft

The most common outcome of "anyone can run code on your fleet" is not a dramatic data breach. It's that your CPU quietly becomes someone else's revenue. A mod that spawns a mining process, or turns your server into a residential proxy exit node, or joins a botnet, is boring, profitable, and hard to notice if you're only alerting on crashes. The tell is usually a support ticket about lag, three weeks late. Community-run servers are an especially juicy target because the operator is a hobbyist with a credit card attached to a cloud account and no egress monitoring.

Reading data that belongs to other players

If your mod runtime shares a process with the game server, the mod is inside your data. Player inventories, chat logs, session tokens, purchase history, the auth secret you loaded into a config object, the database handle sitting in a global. Even a well-intentioned mod API leaks by accident, because the API surface a modder wants (query nearby entities, read player state, persist data) is one careless getter away from the API surface an attacker wants. And on a shard that hosts multiple communities, one mod reading another's world state is a cross-tenant incident with a Discord audience.

Reaching the host filesystem and the host kernel

This is where the dark comedy lives. A modder writes `os.execute("rm -rf ...")` — possibly not even maliciously, possibly as a build helper they forgot to strip — and the question of what happens next is decided entirely by what layer is standing between that string and your production host kernel. If the answer is "we deleted the `os` table from the Lua globals," the answer is really "we hope nobody found the other route to it." There is always another route: a metatable you forgot to freeze, an FFI binding, a native plugin the mod loads, a serialization gadget, a debug library left reachable.

Infinite loops and memory bombs that take the whole shard down

You don't need an exploit to ruin someone's evening. `while true do end` in a tick handler, a recursive world-gen call, an allocation loop, a fork bomb in a native plugin — any of these freeze the shard for every player on it. Most language sandboxes ship an instruction-count hook or a debug hook you can use to interrupt a runaway script, which works right up until the mod is stuck inside a native call that the interpreter can't preempt. Then your options are: kill the process, drop everyone, and explain it in the announcements channel.

Why language-level sandboxes keep losing

A language-level sandbox works by enumerating everything dangerous and removing it: delete `os` and `io` from the Lua environment, strip `require` and `child_process`, replace the global object, hand the script a curated table of safe functions. This is denylisting, and denylisting fails for the usual reason — you have to be right about every single reachable path, forever, across every version of the runtime, while the attacker only has to find one.

The historical record is not ambiguous. Node's `vm` module has always carried an explicit warning that it is not a security mechanism, and `vm2` — the library the ecosystem used for years precisely because people wanted a real boundary — accumulated a long string of sandbox-escape advisories and was eventually deprecated by its own maintainer with the recommendation to use an isolate- or VM-based approach instead. LuaJIT sandboxes have been escaped through the FFI, through bytecode loading, and through metatable tricks. QuickJS and other embeddable engines are smaller targets but still fundamentally share an address space with your host process. Verify the current state of any specific runtime against its own security docs — these projects do fix things — but treat the pattern as the lesson rather than any individual CVE.

There are three structural problems, and none of them are fixed by a better denylist:

• Shared address space. The interpreter, the host game server, and the mod all live in one process. Any escape is immediate and total — no second wall behind the first.
• Native code is the whole point of some mods. The moment a mod needs FFI, a compiled plugin, or a native rendering hook, your language sandbox is not in the conversation at all.
• Resource limits are cooperative. Instruction hooks, allocation counters, and timeouts require the interpreter to be in a position to check them. Native calls, tight C loops, and blocking syscalls are exactly where that assumption fails.

"We removed the io library" describes an API surface, not a security boundary. The boundary is whatever the attacker has to break through to reach your kernel — and in a language sandbox, that's a function pointer away.

A microVM per mod session, or per shard

A Firecracker microVM is a real virtual machine: its own guest kernel, its own memory, its own virtual disk, its own network namespace, confined by hardware virtualization through KVM. It's the same isolation model AWS Lambda uses to run untrusted code from millions of customers. Put the mod runtime — or the whole game shard, mods included — inside one, and the four threat-model entries above change character entirely.

• Mining and proxying → still possible inside the VM, but the VM has a hard vCPU and RAM budget, its own network namespace, and a default-deny egress policy you control at the tap device. The mod can mine against a CPU cap you set, on a network that drops its pool connection.
• Exfiltration → there is nothing to exfiltrate. The VM contains one shard's world state and nothing else. No other community's data, no host credentials, no shared database handle. Total compromise of the guest yields data the players in that shard already had.
• `os.execute` → runs. Genuinely, it runs. It executes against a throwaway guest kernel on a copy-on-write disk that you delete when the session ends. The modder gets a shell in a machine that exists for their benefit and evaporates afterwards.
• Infinite loops and memory bombs → capped at the VM boundary by the host scheduler, not by the interpreter's goodwill. One shard wedges; every other shard on the host doesn't notice.

The objection was always cost and latency: you can't boot a VM per mod session if booting takes seconds and costs a gigabyte. Snapshot-restore is what removes that objection. On PandaStack every create restores a pre-baked snapshot of an already-booted machine rather than booting from scratch — p50 179ms, p99 ~203ms, with the snapshot-restore step itself around 49ms — compared to roughly 3 seconds for the first-ever cold boot before a snapshot exists. Memory is mapped copy-on-write, so a hundred shards running the same server build share the same guest-kernel and binary pages until one of them writes.

Ephemeral sessions vs. persistent shards

Two shapes, chosen by whether the world has to survive. An ephemeral shard — a matchmade round, a minigame lobby, a mod author's test session, a "try this mod" preview link — gets a plain sandbox with a TTL. It boots from the snapshot, hosts the session, and is destroyed. Nothing persists, which is the point: whatever the mod did to that machine dies with it.

A persistent shard — a community's long-running world, a survival server people have been building in for two years — wants a persistent sandbox with a durable volume, so the world data lives on real disk that survives restarts rather than on the ephemeral rootfs. Between play sessions you can hibernate it (snapshot memory and disk, stop the VM) and wake it when the first player connects, so an empty server at 4am costs storage rather than compute. Small communities are the ones who feel that difference most, since they're paying for a box that sits idle most of the week.

Egress control: the mod does not need to call home

Almost every hostile-mod outcome — mining, proxying, exfiltration, pulling a second-stage payload — needs the network. Because each sandbox gets its own network namespace and tap device (PandaStack pre-allocates 16,384 /30 subnets per agent so per-shard networking isn't a bottleneck), egress policy is per-VM and enforced outside the guest's control. Default-deny outbound, allow the coordinator and the game's own services, and a mod's "phone home" turns into a connection timeout. This is the control that language sandboxes structurally cannot offer: they can hide the HTTP client from the script, but they can't stop a native call from opening a socket in the shared process.

Limits inside the guest (defense in depth)

The VM is the boundary; everything below is about making bad mods fail fast and cheaply instead of grinding a shard into the floor. These limits live entirely inside one shard's guest, so tuning them can't affect anyone else, and getting them wrong is a bug rather than an incident.

#!/usr/bin/env bash
# Runs INSIDE one shard's microVM. This is defense in depth, NOT the
# security boundary -- the boundary is the hypervisor. These limits just
# make a runaway or hostile mod fail fast instead of wedging the shard.
set -euo pipefail

MOD_DIR=/srv/mod          # the stranger's code
WORLD_DIR=/srv/world      # this shard's state (durable volume if persistent)
CPU_SECONDS=20            # a tick handler burning 20s of CPU is not a mod
MAX_MEM_MB=512
MAX_FILES=64
MAX_PROCS=64              # fork bombs stop here
WALL_CLOCK=120s

# 1. The mod runs as a nobody user, and its own directory is read-only.
#    Persistence goes through the game server's API, not the filesystem.
id -u modrunner >/dev/null 2>&1 || useradd -r -s /usr/sbin/nologin modrunner
chown -R root:root "$MOD_DIR"
chmod -R a-w "$MOD_DIR"

# 2. Egress: default-deny. A mod has no legitimate reason to reach the
#    internet. No DNS either -- if it can't resolve, it can't call home.
iptables -P OUTPUT DROP
iptables -A OUTPUT -o lo -j ACCEPT
iptables -A OUTPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
iptables -A OUTPUT -d 10.200.0.1 -p tcp --dport 8443 -j ACCEPT  # coordinator

# 3. Run the server (mods loaded) under hard limits. Any breach kills this
#    process. The shard restarts clean from the snapshot; the host and every
#    other shard never notice.
run_shard() {
  ( ulimit -t "$CPU_SECONDS" -v $((MAX_MEM_MB * 1024)) -n "$MAX_FILES" -u "$MAX_PROCS"
    exec timeout --signal=KILL "$WALL_CLOCK" \
      setpriv --reuid=modrunner --regid=modrunner --clear-groups \
      /usr/local/bin/game-server --world "$WORLD_DIR" --mod "$MOD_DIR" )
}

run_shard || echo "mod exceeded its budget; shard will restore from snapshot"

Snapshot the loaded world, fork it per match

Loading a modded world is the slow part of starting a game server, and it's slow in a way that has nothing to do with virtualization: parse the mod manifests, run every mod's init hooks, generate or read the terrain, warm the entity caches, JIT the hot scripts. Doing that once per match instance is pure waste when every instance starts from the same state.

Do it once, snapshot the running machine with the world fully loaded, and fork. A fork is a copy-on-write clone — memory pages mapped MAP_PRIVATE, disk cloned by reflink — so each match instance starts from the identical warmed world and only diverges as players actually change things. On the same host a fork lands in 400–750ms; across hosts it's 1.2–3.5s because the memory image has to move. For a battle-royale-style "forty instances of the same map," that's the difference between forty cold world-loads and one.

from pandastack import Sandbox

def open_match_instances(count: int, mod_pack: str) -> list[Sandbox]:
    """Load the modded world ONCE, then fork it per match instance."""
    base = Sandbox.create(
        template="base",
        persistent=True,
        metadata={"role": "world-template", "mods": mod_pack},
    )

    # The expensive part -- mod init hooks, terrain, entity caches, JIT
    # warmup -- happens exactly once, here.
    load = base.exec(
        f"game-server --load-world /srv/world --mod-pack {mod_pack} --warm",
        timeout_seconds=600,
    )
    assert load.exit_code == 0, load.stderr

    base.snapshot()  # freeze the fully-loaded, mods-initialised world

    # Each fork is copy-on-write: same memory pages, same disk blocks,
    # until a player changes something. Same-host fork: 400-750ms.
    return [
        base.fork(metadata={"match": f"m{i}", "mods": mod_pack})
        for i in range(count)
    ]

The mod author's iteration loop

The same primitive fixes developer experience, which is the thing that usually kills mod-security proposals. Modders will not adopt a sandbox that makes their edit-test loop worse, and historically "secure" meant "wait for a full server restart to see if your change worked." With sub-200ms create, a per-edit test VM is viable: the author saves a file, you push it into a fresh sandbox forked from the warmed world, run a scripted scenario, stream the logs back, and destroy it. Every test starts from a known-clean world, so a mod that corrupts state during testing doesn't poison the next run, and the author never has to remember to reset anything.

The marketplace pipeline: detonate before you publish

If you run a mod marketplace or a Workshop-style catalog, the highest-leverage place to spend isolation is submission time. Static analysis alone is weak — obfuscated Lua and minified JS are trivially able to hide a string — but static analysis plus behavioural observation in a disposable VM catches a lot. Run the submitted mod against a fixture world, watch what it tries to do, throw the machine away, and attach the report to the moderation queue. The mod never touches a machine that matters, and a human reviewer gets evidence instead of a vibe.

from pandastack import Sandbox

# Behaviours that flag a submission before a human reviewer sees it.
RED_FLAGS = [
    "os.execute", "io.popen", "package.loadlib", "require('ffi')",
    "load(", "loadstring(", "child_process", "process.binding",
]


def vet_mod(mod_id: str, archive: bytes) -> dict:
    """Detonate a submitted mod in a throwaway microVM; report behaviour."""
    with Sandbox.create(
        template="base",
        ttl_seconds=600,
        metadata={"purpose": "mod-vetting", "mod": mod_id},
    ) as sbx:
        # 1. Land the untrusted archive inside the guest -- and nowhere else.
        sbx.filesystem.write("/srv/incoming/mod.tar.gz", archive)
        sbx.exec(
            "mkdir -p /srv/mod && tar xzf /srv/incoming/mod.tar.gz -C /srv/mod",
            timeout_seconds=60,
        )

        # 2. Cheap static pass. Not a boundary -- just a triage signal.
        pattern = "|".join(RED_FLAGS).replace("(", "\\(")
        grep = sbx.exec(
            f"grep -REn \"{pattern}\" /srv/mod || true", timeout_seconds=60
        )

        # 3. Detonate against a fixture world under syscall tracing, with
        #    egress already default-denied by the guest's own firewall.
        run = sbx.exec(
            "strace -f -qq -e trace=execve,connect,socket,openat "
            "-o /tmp/syscalls.log -- "
            "timeout 90 game-server --headless --world /srv/fixture "
            "--mod /srv/mod --tick-limit 5000",
            timeout_seconds=180,
        )

        # 4. Collect behaviour: what it spawned, who it dialled, what it burned.
        spawned = sbx.exec(
            "grep -c execve /tmp/syscalls.log || true", timeout_seconds=30
        )
        dialled = sbx.exec(
            "grep -oE 'connect\\(.*sin_addr[^)]*' /tmp/syscalls.log "
            "| sort -u || true",
            timeout_seconds=30,
        )
        blocked = sbx.exec(
            "iptables -L OUTPUT -v -n | awk '/DROP/ {print $1}'",
            timeout_seconds=30,
        )
        cpu = sbx.exec(
            "cat /sys/fs/cgroup/cpu.stat | head -1", timeout_seconds=30
        )

        return {
            "mod": mod_id,
            "exit_code": run.exit_code,
            "static_flags": grep.stdout.splitlines(),
            "processes_spawned": spawned.stdout.strip(),
            "outbound_attempts": dialled.stdout.splitlines(),
            "egress_blocked_packets": blocked.stdout.strip(),
            "cpu": cpu.stdout.strip(),
            "timed_out": run.exit_code == 124,
        }
    # VM destroyed on block exit. Whatever the mod did to it dies here.

A submission that spawns twelve processes, attempts three outbound connections to addresses that aren't yours, and hits the tick limit is not a mod that needs a careful human read — it's a mod that needs a rejection email. A submission that spawns nothing, dials nobody, and exits cleanly still goes to a reviewer, but with a much shorter queue in front of it.

Language sandbox vs. container vs. WASM vs. microVM

Four ways to run somebody else's mod code, from softest to hardest boundary. WASM is the genuinely interesting middle option and has been getting better fast; verify the current isolation and resource-limit story of any specific runtime — Wasmtime, Wasmer, a JS isolate pool, a container runtime — against its own docs rather than this table, because these projects move.

• Escape surface — Language sandbox (Lua/vm2/QuickJS): shares your process address space; escapes are a metatable, FFI binding, or interpreter bug away, with a long public track record. Container: namespaces and cgroups over a shared host kernel — the full Linux syscall surface is the attack surface. WASM: a small, well-specified VM with no ambient authority; escapes are rarer, but host-function bindings are where the bugs live. microVM: hardware virtualization, own guest kernel; an escape requires a hypervisor break.
• Resource limits — Language sandbox: cooperative (instruction hooks, allocation counters) and unenforceable once you're inside a native call. Container: cgroup CPU/memory limits, real but sharing page cache and kernel with neighbours. WASM: fuel/epoch interruption and a linear memory cap — genuinely good, per-instance. microVM: hard vCPU and RAM caps at the VM boundary, enforced by the host scheduler regardless of what the guest is doing.
• Native-code mods — Language sandbox: no (a native plugin is the escape). Container: yes, but that native code runs against your host kernel. WASM: only if the mod compiles to WASM; existing native plugin ecosystems don't port for free. microVM: yes, arbitrary native code, arbitrary syscalls, arbitrary binaries — the guest is just Linux.
• Startup cost — Language sandbox: microseconds; a new interpreter state is nearly free. Container: milliseconds to seconds depending on image pull and runtime init. WASM: sub-millisecond instantiation, which is its headline strength. microVM: sub-second — 179ms p50 on PandaStack via snapshot-restore (the restore step ~49ms), versus ~3s for a cold boot with no snapshot.
• Dev ergonomics for modders — Language sandbox: familiar until they hit a removed API and file an issue. Container: normal Linux, but the modder must produce an image. WASM: a real toolchain shift and a restricted std library; great for new ecosystems, painful for existing ones. microVM: it's a whole Linux box — existing mods, existing tooling, existing debuggers, no porting.
• Best fit — Language sandbox: trusted first-party scripting and prototyping. Container: your own build and CI steps. WASM: greenfield plugin APIs where you control the toolchain and want per-call isolation. microVM: untrusted third-party mods, community shards, marketplace vetting, anything with native code.

The honest summary: WASM and microVMs are both defensible answers, and they're good at different things. WASM wins when you're designing a new plugin API from scratch, want thousands of tiny isolated calls per second, and can require modders to use your toolchain. microVMs win when the mods already exist, already assume a filesystem and a process model and native libraries, and you need the boundary to hold against code you will never review.

When a microVM per mod is the wrong call

Don't reach for this if your mods are first-party. If the only people writing scripts are your own designers, a Lua state with a curated API is faster to build, faster to run, and completely adequate — the threat model is "a colleague made a mistake," and a code review handles that. Don't reach for it for per-call plugin hooks either: if your architecture invokes a mod function thousands of times per second per player, a VM boundary is the wrong granularity and you want WASM or an in-process isolate. And if you already have a mature WASM plugin ecosystem where every mod compiles to your toolchain, adding VMs underneath is belt-and-braces you probably don't need yet.

The microVM earns its place when the code comes from strangers, the mods carry native code or expect a real filesystem, one shard's failure must not touch another's, and you'd like the answer to "what if a mod calls `os.execute`?" to be "then it runs, in a machine we throw away" rather than a security review. That used to be an expensive answer. With snapshot-restore create at p50 179ms, copy-on-write forks of a warmed world in 400–750ms on the same host, and hibernated shards costing storage instead of compute, it's now mostly just the correct one — and considerably cheaper than the incident report you'd otherwise be writing at 2am while a mod mines somebody else's coins on your fleet.