How to Build a Remote Code Execution API Safely

Sooner or later you build the endpoint: `POST /run` takes a blob of user code, executes it, and returns the output. A code playground, an autograder, an LLM tool that runs model-written snippets, a spreadsheet-formula evaluator, a low-code backend that lets customers write hooks. The feature is the same in every case, and so is the honest name for it — you are building a remote code execution API. RCE is the single most severe class of vulnerability there is, and here you are shipping it as a product. That's fine. It's a legitimate, valuable feature. But it only works if the code runs somewhere it can't hurt you, and "somewhere" has exactly one correct answer at production scale: a fresh Firecracker microVM per request.

I'm Ajay — I built PandaStack, which is a microVM sandbox platform, so I have skin in this game. But the argument here doesn't depend on which platform you pick. It depends on where the isolation boundary sits. This post walks the threat model of an RCE endpoint, why every cheaper boundary (exec, subprocess, containers) fails against it, and the microVM-per-request pattern in enough detail to build it: resource and time limits, network egress control, and how you actually read results back out.

The threat model: assume the code is hostile

The mistake is designing for the code you expect and hoping for the best on the rest. Design for the worst input you could receive, because eventually you will receive it — from an attacker probing your API, from a prompt-injected LLM, or from an ordinary user who pasted something they didn't understand. The submitted code is not a snippet that computes fibonacci. It is an adversary with a shell, and your job is to give it a shell inside a box you're willing to set on fire.

Concretely, assume any request may try to do all of the following:

• Read your secrets — `os.environ`, the instance metadata endpoint (169.254.169.254), mounted credential files, /proc, the process list of neighbors.
• Exfiltrate data — open an outbound socket and POST whatever it found to an attacker-controlled host.
• Exhaust resources — fork bombs, a 40 GB allocation, an infinite `while True`, filling the disk, pinning every core. A denial of service against the box is trivially cheap to write.
• Escape the boundary — a kernel exploit, a container breakout, a symlink trick out of a chroot. If two tenants share a kernel, one hostile tenant can reach the other.
• Persist and pivot — write a cron job, leave a reverse shell, poison a cached artifact so the next tenant's run is compromised.
• Just be malicious for fun — run `rm -rf /` and see what happens. On a shared host, what happens is everyone's day gets ruined.

Why exec(), subprocess, and containers all fail

The tempting boundaries are the ones already sitting in your process. They are also the ones with no boundary at all.

exec() / eval() in your own process

Running untrusted code with `exec()` or `eval()` in your application process is not sandboxing — it's inviting the code into your address space. It shares your memory, your file descriptors, your database connection, your secrets, and your uptime. People reach for "restricted" execution (stripping builtins, a custom `__globals__`) and it has been broken every single time; Python's object graph always has a path back to `os` through some dunder attribute. The correct number of untrusted lines to run in your own interpreter is zero.

A subprocess under a locked-down OS user

A subprocess is better — at least it's a separate process — but it still runs on your host kernel, sees your filesystem (a chroot is not a security boundary), can reach your network, and shares the machine with everything else. You can pile on seccomp, cgroups, namespaces, and a read-only mount and get somewhere real, but at that point you are hand-rolling a container runtime, and you'll get a detail wrong. The failure mode is silent: it looks like it works right up until the input that escapes it.

A container per request

This is where most teams stop, and it's the most dangerous stopping point because it feels finished. A container has namespaces and cgroups and looks isolated. But every container on a machine shares one host kernel, and the Linux syscall interface is an enormous, ever-changing attack surface. Container escapes are a recurring, well-documented class of bug — not an exotic edge case. A container is a polite suggestion to the kernel about who should see what; a kernel bug or a misconfigured capability turns the suggestion into a full host compromise, and with it every other tenant on the box. Containers are the right tool for your own trusted services. They are the wrong trust boundary for code an adversary wrote.

The pattern: a microVM per request

A microVM gives each run its own guest kernel under hardware virtualization (KVM). Firecracker is the VMM AWS Lambda and Fargate use to run untrusted, multi-tenant code exactly this way, which is a strong existence proof that the model works at scale. The historical objection was speed — nobody wanted to cold-boot a VM per request. Snapshot-restore killed that objection: instead of booting a kernel, you restore a pre-booted, pre-warmed snapshot. On PandaStack the restore step itself is ~49 ms and an end-to-end create is p50 179 ms, p99 ~203 ms. VM-grade isolation now costs about what a container cold start used to.

The pattern for an RCE endpoint is deliberately boring: one request, one fresh VM, destroyed at the end. No reuse across tenants, ever.

1. Request arrives with a code payload (and optionally stdin, a language, files).
2. Create a fresh sandbox from a baked template — a new microVM, isolated kernel, empty of any prior run's state.
3. Write the code into the guest filesystem and exec it with a hard timeout.
4. Capture stdout, stderr, and the exit code; read back any output files the run produced.
5. Destroy the sandbox. Its memory, disk, and network namespace go with it.
6. Return the result. The blast radius of whatever just ran was one disposable VM.

Because a create is a snapshot restore rather than a boot, per-request VMs are actually affordable. If you want every run to start from a specific post-setup state (a dataset loaded, a package installed), snapshot a prepared sandbox once and fork it: a same-host fork is 400–750 ms and shares memory copy-on-write, so you get warm state without a warm-pool of shared, reusable-across-tenants VMs. Cross-host forks run 1.2–3.5 s if the scheduler places the child on another agent.

The execution loop in code

Here's the core of it with the Python SDK. Create a sandbox, write the untrusted code to a file, exec it with a timeout, capture the streams, and let the context manager destroy the VM on the way out. Set `PANDASTACK_API_KEY` in your environment; the base URL defaults to https://api.pandastack.ai.

from pandastack import Sandbox

def run_untrusted(code: str, stdin: str = "") -> dict:
    """Run one untrusted snippet in a throwaway microVM and return the result."""
    # ttl_seconds is a backstop: if we crash before killing the VM, it reaps itself.
    with Sandbox.create(template="code-interpreter", ttl_seconds=120) as sbx:
        # Write the caller's code into the guest, never into a shell string.
        sbx.filesystem.write("/workspace/main.py", code)
        if stdin:
            sbx.filesystem.write("/workspace/stdin.txt", stdin)
            cmd = "python3 /workspace/main.py < /workspace/stdin.txt"
        else:
            cmd = "python3 /workspace/main.py"

        # timeout_seconds is the circuit breaker for infinite loops.
        result = sbx.exec(cmd, timeout_seconds=30)

        return {
            "exit_code": result.exit_code,
            "stdout": result.stdout,
            "stderr": result.stderr,
            "duration_ms": result.duration_ms,
        }
    # microVM is destroyed here — memory, disk, and network namespace gone


print(run_untrusted("print(sum(range(100)))"))
# {'exit_code': 0, 'stdout': '4950\n', 'stderr': '', 'duration_ms': ...}

Two non-negotiables live in that snippet. Write the code to a file rather than interpolating it into a shell command — otherwise you've added a shell-injection bug on top of the RCE you meant to build, and multi-line code turns quoting into a minefield. And always pass both a `timeout_seconds` on exec and a `ttl_seconds` on create: the first bounds a single runaway command, the second guarantees the VM dies even if your own process falls over before it can clean up.

Wrapping it as a real HTTP endpoint

The endpoint itself is thin — it exists to validate input, enforce limits, call the sandbox, and shape the response. All the actual danger is handled by the fact that the code runs in a VM you're about to throw away. Here's a FastAPI version with the guardrails that matter in production.

from fastapi import FastAPI, HTTPException
from pydantic import BaseModel, Field
from pandastack import Sandbox

app = FastAPI()

MAX_CODE_BYTES = 256 * 1024   # reject absurd payloads before spending a VM
EXEC_TIMEOUT_S = 30           # per-run wall clock
SANDBOX_TTL_S = 120           # backstop reaper


class RunRequest(BaseModel):
    code: str = Field(max_length=MAX_CODE_BYTES)
    stdin: str = ""


@app.post("/v1/run")
def run(req: RunRequest):
    if not req.code.strip():
        raise HTTPException(400, "empty code")

    # One request -> one fresh microVM -> destroyed on block exit. No reuse.
    with Sandbox.create(template="code-interpreter", ttl_seconds=SANDBOX_TTL_S) as sbx:
        sbx.filesystem.write("/workspace/main.py", req.code)
        cmd = "python3 /workspace/main.py"
        if req.stdin:
            sbx.filesystem.write("/workspace/stdin.txt", req.stdin)
            cmd += " < /workspace/stdin.txt"

        r = sbx.exec(cmd, timeout_seconds=EXEC_TIMEOUT_S)
        return {
            "exit_code": r.exit_code,
            "stdout": r.stdout[-64_000:],   # cap what you echo back
            "stderr": r.stderr[-64_000:],
            "duration_ms": r.duration_ms,
        }

Note what's NOT in the endpoint: no allowlist of imports, no AST inspection, no attempt to reason about what the code will do. That's the point. You don't have to predict the code's behavior because it can't reach anything that matters. The endpoint's own job is the mundane stuff — rejecting oversized payloads before you spend a VM on them, truncating output so a `print` loop can't blow up your response, and applying an auth/rate limit in front so a single caller can't spin up VMs without bound.

Resource and time limits

Isolation stops the code from reaching your stuff. Limits stop it from denying service by consuming everything inside its own box (and running up your bill). A microVM helps here structurally: memory and vCPU are fixed at snapshot-restore time, so a guest physically cannot allocate more RAM than the template baked in — a 40 GB `malloc` fails inside the VM instead of paging out your host. Layer these limits:

• Wall-clock timeout — `timeout_seconds` on every exec. This is your defense against infinite loops and is the single most important limit. Nothing runs forever.
• Memory / vCPU ceiling — bounded by the template's baked snapshot size, so the guest can't exceed it no matter what the code does. Pick a template sized to your workload.
• Sandbox TTL — `ttl_seconds` on create reaps a VM you forgot (or crashed before killing). Defense in depth against leaked VMs.
• Payload + output caps — reject huge code inputs up front; truncate stdout/stderr before returning so the response itself can't be the DoS.
• Concurrency / rate limit — cap VMs-per-caller at your API gateway. A single agent host holds 16,384 pre-allocated /30 subnets, so slots aren't the bottleneck — memory is, and unbounded concurrency will find it.
• Disk — the CoW rootfs is throwaway, but bound the run's time so it can't fill the guest disk indefinitely; the VM's destruction reclaims it regardless.

Network egress control (the exfiltration problem)

The isolation boundary keeps hostile code out of your host. It does not, by default, stop that code from reaching the internet — and "read a secret, then POST it to attacker.example" is the whole exfiltration playbook. Network egress from the guest is real and must be controlled at the network layer, because you cannot trust the code to control itself.

Decide the egress posture per use case, from strictest to loosest:

• No network — the safest default for pure compute (autograders, formula evaluation). If the code has no reason to reach out, don't let it. Nothing to exfiltrate to.
• Allowlist only — permit a small set of destinations (a package mirror, a specific API) and drop everything else, enforced by the netns firewall rules, not by the guest.
• Metadata endpoint blocked — even with network on, always block 169.254.169.254 and link-local ranges so the code can't read cloud instance credentials. This is table stakes.
• Open egress — only when the workload genuinely needs arbitrary outbound (a scraping agent) and you've accepted that the code can send anything it reads to anywhere.

Reading results back out

An RCE API is only useful if you can get the answer back. There are two channels. For anything textual, exec already hands you `stdout`, `stderr`, and `exit_code` — check the exit code first, because a non-zero exit almost always means the interesting output was never produced. For richer artifacts (a rendered chart, a generated file, a large structured result), have the code write to a known path and read the bytes back through the filesystem API before the VM is destroyed.

from pandastack import Sandbox

report_code = """
import json
result = {"rows": 42, "ok": True, "summary": "processed"}
with open("/workspace/out.json", "w") as f:
    json.dump(result, f)
print("done")
"""

with Sandbox.create(template="code-interpreter", ttl_seconds=120) as sbx:
    sbx.filesystem.write("/workspace/job.py", report_code)
    r = sbx.exec("python3 /workspace/job.py", timeout_seconds=30)

    # Gate artifact reads on a clean exit — no exit 0, no file.
    if r.exit_code == 0:
        out = sbx.filesystem.read("/workspace/out.json")  # raw bytes
        with open("result.json", "wb") as f:
            f.write(out)
        print(f"pulled {len(out)} bytes of structured output")
    else:
        print("run failed:", r.stderr)

The structured-file channel scales better than scraping stdout for anything beyond a line or two: have the code emit JSON to `/workspace/out.json` and parse that on the host, rather than trying to reverse-engineer meaning out of mixed print statements. The same pattern reads back a plot PNG, a CSV, an Excel export, or an audio clip — write a known path in the guest, confirm exit code 0, then `filesystem.read` it before the sandbox is destroyed.

Boundaries side by side

• Shared kernel — exec/subprocess: yes, the code IS your process or its child. Container: yes, one host kernel for all tenants. MicroVM: no, each guest has its own kernel.
• Escape surface — exec/subprocess: none needed, it's already inside. Container: the full Linux syscall interface. MicroVM: the hypervisor (small, heavily audited).
• Resource DoS containment — exec/subprocess: takes your host down. Container: cgroups help but share the kernel scheduler. MicroVM: fixed RAM/vCPU, a bad alloc fails inside the guest.
• Multi-tenant safety — exec/subprocess: none. Container: one escape reaches every neighbor. MicroVM: one hostile tenant is contained to its VM.
• Startup cost — exec/subprocess: ~0. Container: ms to seconds (image + runtime init). MicroVM: ~49 ms restore step, p50 179 ms create via snapshot-restore.
• Correct use — exec/subprocess: only code you wrote and trust. Container: your own first-party services. MicroVM: untrusted, user-, or model-generated code.

Describe competitors qualitatively and verify the numbers against their own docs — startup times and isolation models change. The load-bearing point isn't a benchmark, it's the first row of that table: only the microVM removes the shared kernel, and for an endpoint whose entire purpose is running an adversary's code, the shared kernel is the whole game.

When a microVM is overkill

Be honest with yourself about the trust boundary. If the code is yours — a plugin from a vetted first-party team, an internal migration script, a well-reviewed dependency — a subprocess or container is simpler, faster, and completely appropriate. The microVM-per-request pattern earns its keep specifically when the code is untrusted, multi-tenant, or generated at runtime by a model or a stranger on the internet. That's the moment "it's just a POST that runs some code" quietly becomes "I've published a remote code execution endpoint," and the moment a disposable VM stops being paranoia and starts being the only responsible design.