Isolating an AI Code-Review Bot That Runs Untrusted PR Code

An AI code-review bot that only reads the diff is a linter with a personality. The useful ones do more: they check out the branch, run `npm install`, build it, run the test suite, sometimes execute the changed code to see what it actually does before leaving a comment. That last part is the whole value — an LLM reviewer that runs the code gathers real evidence instead of guessing from text. It's also the part that quietly turns your CI into a remote-code-execution endpoint for anyone on the internet.

Because when the pull request comes from a fork — an external contributor, a first-time drive-by, a bot account someone spun up ten minutes ago — you are running code you did not write, cannot fully review, and did not consent to line by line. A forked PR is a stranger handing you a USB stick and asking you to run the contents as root, on the same machine that holds your deploy keys. This post is about doing that safely: why a shared runner or container is the wrong boundary, and how a fresh Firecracker microVM per PR contains the blast radius. I'm Ajay, I built PandaStack — I'll be honest about where the boundary actually is.

The attack: a malicious PR against a shared runner

Picture a normal setup. Your review bot listens for pull requests, checks out the head ref, and runs the project's build and tests on a long-lived CI runner or a container that gets recycled between jobs. Convenient, fast, and — for untrusted PRs — a gift to an attacker. Here is what a hostile contributor can do without ever touching your review comments:

• Exfiltrate CI secrets. The runner's environment holds tokens — a registry password, a cloud credential, a bot's GitHub token. A malicious `postinstall` script or a one-line addition to the test setup can read every env var and POST it to an attacker's server before your bot writes a single word.
• Poison the build cache. Shared runners reuse dependency caches and Docker layers across jobs. A PR that writes a backdoored artifact into the cache path gets that artifact executed in the next legitimate job — including one on your trusted main branch.
• Escape to the host. If the runner is a plain container, the PR's code shares the host kernel with every other job and the host itself. One container-escape bug — a known, recurring class — and the attacker owns the machine, the neighboring PRs, and whatever that host can reach.
• Pivot through the network. From inside the runner, untrusted code can hit your cloud metadata endpoint, your internal package registry, or a database that happened to be reachable, harvesting more credentials as it goes.
• Burn your quota and bury the evidence. Crypto miners in PRs are a whole genre. A busy loop or a 40 GB allocation takes the shared runner — and every job queued behind it — down with it.

None of this requires a sophisticated adversary. The whole exploit is often three lines in `package.json` or a `conftest.py`. The reason it works is that the review bot's convenience — reusing a warm runner, sharing a cache, keeping secrets handy for posting comments — is exactly the surface the attacker needs.

Why "just run it in a container" isn't enough

The instinct is right — isolate the untrusted code — but a plain container is the wrong boundary for it. A container is a Linux process with a private view of the system (namespaces) and a resource budget (cgroups), running on the host's shared kernel. Every container on the machine is talking to the same kernel. That kernel is the entire Linux syscall surface, and container escapes through it are a known, recurring class of bug rather than a hypothetical. For code you wrote and trust, that's fine. For a forked PR from an account created this morning, you're betting the host on the assumption that this particular stranger didn't bring a kernel exploit.

gVisor and Kata Containers exist precisely because plain containers aren't enough here — gVisor intercepts syscalls in a user-space kernel to shrink the surface, Kata wraps the container in a lightweight VM. They're real options, and they're all quietly conceding the same point: a shared host kernel is not a boundary you want to bet an external contributor's code against.

A Firecracker microVM is a different category. It boots its own guest kernel under hardware virtualization (KVM) and can only touch the outside world through a tiny set of emulated virtio devices. There is no shared kernel to attack — an escape would have to break the hypervisor itself, a vastly smaller and more heavily audited surface than the full syscall interface a container sees. This is the same VMM AWS Lambda uses to run untrusted code from millions of customers on shared fleets. The blast radius of a malicious PR becomes exactly one disposable VM.

The model: one fresh microVM per pull request

The design that makes untrusted-PR execution safe is boringly simple to state: every pull request run gets its own fresh microVM, created empty of secrets, destroyed the moment the run finishes. No reuse across PRs, no shared cache mounted from the host, no long-lived credentials in the guest environment. The VM is the trust boundary, so the rule is one trust domain per VM — and a forked PR is its own trust domain.

Historically the objection was latency: nobody wants to cold-boot a full VM per PR. That's what makes microVM snapshots the enabling trick. PandaStack creates each sandbox by restoring a baked snapshot on demand rather than cold-booting — the restore step is around 49ms, and a create is p50 179ms (p99 ~203ms). Only the very first spawn of a template pays the ~3s cold boot; after that every PR gets a VM in well under a quarter second. VM-grade isolation stops costing you a coffee break per run.

Wiring it up: check out the PR, build it, gather evidence

Here's the core of a review bot in the PandaStack Python SDK. It creates a throwaway sandbox on the base template, writes the PR's checkout script and the untrusted diff into the guest, runs the build and tests with a timeout, and returns structured evidence. Crucially, the sandbox is created with no secrets in its environment — the only thing it can see is the code under review.

from pandastack import Sandbox

def review_pull_request(repo_url: str, head_sha: str) -> dict:
    """Run an untrusted forked PR's build + tests in a throwaway microVM.
    No CI secrets are ever placed in the guest environment.
    """
    # Fresh VM per PR. ttl_seconds is a backstop in case we forget to kill it.
    with Sandbox.create(template="base", ttl_seconds=900) as sbx:
        # Shallow-clone only the PR head. The clone runs INSIDE the VM, so a
        # malicious repo URL or git hook can't touch the host.
        clone = sbx.exec(
            f"git clone --depth 1 {repo_url} /workspace/pr && "
            f"cd /workspace/pr && git fetch --depth 1 origin {head_sha} && "
            f"git checkout {head_sha}",
            timeout_seconds=120,
        )
        if clone.exit_code != 0:
            return {"status": "checkout_failed", "log": clone.stderr}

        # Install + build + test. This is the untrusted code executing.
        # If it mines crypto or forkbombs, it does so in a VM we're about
        # to delete, with no credentials to steal.
        build = sbx.exec(
            "cd /workspace/pr && npm ci && npm run build && npm test",
            timeout_seconds=600,
        )

        # Pull structured evidence back out for the LLM reviewer to read.
        return {
            "status": "ok" if build.exit_code == 0 else "failed",
            "exit_code": build.exit_code,
            "stdout": build.stdout[-8000:],  # tail, for the model's context
            "stderr": build.stderr[-8000:],
            "duration_ms": build.duration_ms,
        }
    # microVM is incinerated here — nothing the PR did survives.

The AI-agent angle drops in on top of this. The LLM reviewer doesn't run the code itself; it calls `review_pull_request` as a tool, reads back the real exit codes, test output, and any artifacts, and then writes its comment grounded in what actually happened rather than in a hopeful reading of the diff. "This PR claims to fix the race condition" becomes "I ran the suite; the flaky test still fails 3 in 10 — here's the output." The model gets evidence; the untrusted code never gets your host.

For richer probes, have the PR's own code write a result file and read it back out. The filesystem API returns raw bytes, so a coverage report, a screenshot from an e2e test, or a JSON summary the changed code emitted all come back the same way:

with Sandbox.create(template="base", ttl_seconds=900) as sbx:
    # ... clone + build as above ...

    # Ask the PR's test harness to emit a machine-readable summary.
    sbx.filesystem.write(
        "/workspace/pr/probe.sh",
        "cd /workspace/pr && npm test -- --reporter=json "
        "> /workspace/pr/report.json 2>/dev/null || true\n",
    )
    sbx.exec("bash /workspace/pr/probe.sh", timeout_seconds=300)

    # Read the artifact back as bytes; feed the parsed result to the LLM.
    report = sbx.filesystem.read("/workspace/pr/report.json")
    evidence = report.decode("utf-8", errors="replace")

Close the network, or the glovebox has a window

Isolation contains the kernel; it does not automatically contain the network. By default a sandbox can reach the internet, which is fine for `npm install` but is also the exact door an attacker uses to POST your (now absent) secrets outward or to pull down a second-stage payload. If the code you run is genuinely untrusted, the network is part of the threat model, not an afterthought.

Two useful postures. If the PR needs to install dependencies, allow egress only to your package registry and deny the rest. If you can vendor dependencies or pre-bake them into the template, run the build with egress fully off — the cleanest possible answer, since untrusted code with no network can't exfiltrate anything or phone home. On PandaStack each sandbox gets its own network namespace (one of 16,384 pre-allocated /30 subnets per agent), so egress policy is per-VM rather than a shared host firewall you have to reason about globally. A sketch of a deny-by-default posture at the guest boundary:

# Run inside the microVM before executing untrusted PR code.
# Deny all egress by default, then allow ONLY the package registry.
# (Adjust the registry IP/CIDR to your mirror; DNS to it must be allowed too.)
iptables -P OUTPUT DROP
iptables -A OUTPUT -o lo -j ACCEPT
iptables -A OUTPUT -d 10.0.0.0/8 -j DROP          # no internal-network pivots
iptables -A OUTPUT -p udp --dport 53 -j ACCEPT    # DNS (to your resolver)
iptables -A OUTPUT -d 203.0.113.10 -p tcp --dport 443 -j ACCEPT  # registry
# Everything else — attacker C2, metadata endpoints, exfil — is dropped.

Side by side: shared runner vs microVM-per-PR

• Isolation boundary — Shared runner: namespaces + cgroups on a host kernel shared with every job. microVM-per-PR: hardware-virtualized guest with its own kernel, one PR per VM.
• Secret exposure — Shared runner: CI tokens live in the same environment the untrusted build runs in. microVM-per-PR: guest is created credential-free; the posting token stays on the host.
• Cache poisoning — Shared runner: dependency and layer caches persist across jobs and can be tainted. microVM-per-PR: nothing survives the run; each PR starts from the same clean baked snapshot.
• Escape blast radius — Shared runner: one container escape reaches the host and every neighboring job. microVM-per-PR: an escape would have to break the hypervisor; damage is capped at one disposable VM.
• Cleanup — Shared runner: you hope the recycle step scrubbed everything the last PR touched. microVM-per-PR: you delete the VM, so cleanup is total by construction.
• Cost of a fresh env per PR — Shared runner: reuse is fast but that reuse is the vulnerability. microVM-per-PR: snapshot-restore create is ~49ms (p50 179ms), so a fresh VM per PR is cheap enough to be the default.
• Verify against docs — This table describes PandaStack's model and Firecracker's isolation properties; for any other platform (gVisor, Kata, hosted CI), confirm the specifics against their own documentation before relying on them.

When you don't need a microVM

Be honest about the threat model. If every PR comes from a teammate with commit access to the repo, the code isn't really untrusted — a plain runner is simpler and the isolation is theater. The microVM boundary earns its keep the moment forked or first-time-contributor PRs can trigger execution, which for any public repo or open-source project is essentially always. The tell is simple: if a stranger's push can cause your infrastructure to run their code, treat that code as hostile and give it its own VM.

The payoff is that you get to have the good version of the review bot — the one that actually runs the code and reviews reality — without the version where a forked PR quietly walks off with your deploy keys. Put the stranger's USB stick in a machine you're about to throw in the incinerator, read the results through the glass, and never let it near the one holding your secrets.