How to run Playwright tests in a sandbox

Playwright is a well-engineered piece of software that behaves badly in a specific environment: a shared CI runner with two cores, four gigabytes of memory, and a 64 MB `/dev/shm`. That is not Playwright's fault. Chromium is a memory-hungry multi-process program, and running eight of them on a machine sized for a unit-test suite produces timeouts that get labelled flakiness and retried until they pass.

This walks through running a suite across isolated microVMs instead — one shard per machine, each with its own kernel, its own page cache, and its own shared memory. The mechanics are simple; the sizing and the artifact handling are where the useful detail is.

What isolation actually buys you

Three things, and it is worth separating them because only two are about isolation.

• Shared memory. Chromium moves rendered frames between its processes through `/dev/shm`, and the container default of 64 MB is far below what it needs. When it runs out, tabs crash and the driver reports a lost connection, which surfaces as a test failure blaming your application. In a microVM, `/dev/shm` is the guest's own tmpfs — half of the guest's RAM by default — so on a 4 GiB guest you get 2 GiB of it and the problem does not exist.
• No cross-shard interference. On a shared runner, shard 3's page cache eviction is shard 1's slow page load. Each shard having its own kernel means one shard's memory pressure is genuinely invisible to the others, so a timeout means something.
• Throughput. This is not isolation, it is just parallelism — but it is usually the reason anyone starts. Ten machines running one shard each finish in roughly a tenth of the time, and a fifty-minute suite becomes a five-minute one.

Start with one shard

Get a single shard working end to end before you fan out. Clone inside the sandbox rather than uploading the repository — it is faster, and nothing crosses your process.

import shlex
from pandastack import Sandbox

REPO = "https://github.com/your-org/your-app.git"
COMMIT = "a1b2c3d"

sbx = Sandbox.create(
    template="browser",       # Chromium + Playwright already installed
    ttl_seconds=2400,         # generous: a TTL that fires mid-suite is a
)                             # confusing failure. Bound it, don't remove it.

try:
    setup = sbx.commands.run(
        f"git clone --filter=blob:none {shlex.quote(REPO)} /work/repo "
        f"&& cd /work/repo && git checkout -q {shlex.quote(COMMIT)} "
        "&& npm ci --prefer-offline --no-audit",
        timeout=900,
    )
    assert setup.exit_code == 0, setup.stderr

    # --workers=3, not 8. Reasoning in the next section: this is a memory
    # ceiling, not a CPU one, and 8 vCPUs will happily let you OOM.
    run = sbx.commands.run(
        "cd /work/repo && npx playwright test "
        "--workers=3 --reporter=list,html --trace=retain-on-failure",
        timeout=1800,
    )
    print(run.stdout[-4000:])
finally:
    # Artifacts first, teardown second -- always in a finally, because the
    # failing run is the one whose trace you need.
    if sbx.filesystem.exists("/work/repo/playwright-report"):
        sbx.commands.run(
            "tar czf /tmp/report.tgz -C /work/repo playwright-report test-results",
            timeout=300,
        )
        sbx.filesystem.download("/tmp/report.tgz", "./report.tgz")
    sbx.kill()

Worker count is a memory decision, not a CPU one

This is the single most common mistake. Playwright defaults to roughly half the available cores, and on an 8-vCPU guest that suggests four workers — but each worker runs its own Chromium, and a Chromium rendering a modern application comfortably occupies several hundred megabytes across its browser, renderer, and GPU processes. Multiply by workers, add Node and the test framework, and 4 GiB disappears faster than you expect.

What happens when you overcommit is not a clean error. The guest's OOM killer picks a process, which is usually a renderer, and Playwright reports a page crash or a timeout on whatever test happened to be running. So the failure is attributed to a test rather than to your worker count, and it moves around between runs. If your flakiness has no pattern except that it gets worse with parallelism, this is where to look.

# Measure rather than guess. Run the suite with a worker count and watch
# the guest's memory while it goes.
#
# In one exec session:
npx playwright test --workers=4
#
# In another, against the same sandbox:
while true; do
  free -m | awk 'NR==2 {printf "used=%sMB avail=%sMB\n", $3, $7}'
  sleep 2
done

# If "avail" approaches zero at any point, you are one heavy page away
# from a renderer being killed and a test being blamed for it.
#
# Rule of thumb for a 4 GiB guest: 2-3 workers for an app-heavy suite,
# 4 for a light one. Verify /dev/shm is not the constraint too:
df -h /dev/shm      # microVM guest: ~2G on a 4 GiB guest. Container: 64M.

Then shard across machines

Playwright's own `--shard=i/n` splits the suite deterministically by test file, so each machine runs a disjoint subset with no coordination. That makes the fan-out embarrassingly parallel: create N sandboxes, give each one its shard index, collect the results.

The one thing that needs care is merging the reports. Each shard produces its own blob, and Playwright has a merge step specifically for this — use it, rather than looking at N separate HTML reports and trying to reason about the whole run.

import shlex
from concurrent.futures import ThreadPoolExecutor
from pandastack import Sandbox

SHARDS = 8
REPO = "https://github.com/your-org/your-app.git"
COMMIT = "a1b2c3d"
BASE_URL = "https://staging.example.com"

def run_shard(i: int) -> dict:
    sbx = Sandbox.create(
        template="browser",
        ttl_seconds=2400,
        metadata={"suite": "e2e", "shard": str(i), "commit": COMMIT},
    )
    try:
        sbx.commands.run(
            f"git clone --filter=blob:none {shlex.quote(REPO)} /work/repo "
            f"&& cd /work/repo && git checkout -q {shlex.quote(COMMIT)} "
            "&& npm ci --prefer-offline --no-audit",
            timeout=900,
        )
        # 'blob' reporter emits a mergeable report per shard. Do NOT use
        # 'html' per shard -- you cannot meaningfully combine those.
        r = sbx.commands.run(
            f"cd /work/repo && PLAYWRIGHT_BASE_URL={shlex.quote(BASE_URL)} "
            f"npx playwright test --shard={i + 1}/{SHARDS} "
            "--workers=3 --reporter=blob --trace=retain-on-failure",
            timeout=1800,
        )
        # Pull the blob out whatever happened -- a failed shard's report is
        # the one worth having.
        sbx.filesystem.download(
            "/work/repo/blob-report/report.zip", f"./blob/shard-{i}.zip"
        )
        return {"shard": i, "ok": r.exit_code == 0, "tail": r.stdout[-2000:]}
    finally:
        sbx.kill()

with ThreadPoolExecutor(max_workers=SHARDS) as pool:
    results = list(pool.map(run_shard, range(SHARDS)))

failed = [r["shard"] for r in results if not r["ok"]]
for r in results:
    if not r["ok"]:
        print(f"--- shard {r['shard']} ---\n{r['tail']}")

# One report for the whole run, locally:
#   npx playwright merge-reports --reporter=html ./blob
if failed:
    raise SystemExit(f"shards failed: {failed}")

Getting artifacts out, and looking at them

Traces are the reason a failing browser test is debuggable at all: a Playwright trace holds the DOM snapshots, network log, console output, and a timeline you can step through. They are also large, which is why `--trace=retain-on-failure` is the right default — full tracing on every test produces gigabytes you will never open.

Two ways to look at them. Download the blob reports and merge locally, which is what you want in CI. Or, when you are debugging interactively, serve the report from inside the sandbox and open it in your browser — every sandbox port is reachable at a preview URL for the sandbox's lifetime, with no tunnel to set up.

# Debugging interactively: serve the HTML report from the sandbox and
# open it, instead of downloading and unpacking an archive.
sbx.commands.run(
    "cd /work/repo && setsid nohup npx playwright show-report "
    "--host 0.0.0.0 --port 9323 > /tmp/report.log 2>&1 &",
    timeout=60,
)

print(sbx.preview_url(9323))
# -> https://9323-<sandbox-id>.pandastack.ai
#
# Reachable for the sandbox's lifetime with no auth: the sandbox UUID IS
# the credential. Fine for a trace you are about to look at; do not paste
# it into a public issue, and let the sandbox expire when you are done.

# Same trick for the trace viewer on a single trace:
#   npx playwright show-trace --host 0.0.0.0 --port 9324 \
#     test-results/<test>/trace.zip

Headed mode and xvfb, if you need it

Headless Chromium is not identical to headed Chromium — extensions do not load, some media paths differ, and a small number of rendering behaviours diverge. When a test only reproduces headed, you need a display, and the `browser` template ships xvfb for exactly that.

# A virtual display, for the tests that only fail with a real one.
# xvfb-run handles server startup, a free display number, and teardown.
xvfb-run --auto-servernum --server-args="-screen 0 1920x1080x24" \
  npx playwright test --headed --workers=1

# --workers=1 with --headed is deliberate: headed Chromium costs
# noticeably more memory than headless, and this is the mode where
# overcommitting bites hardest. Use it for the specific failing spec,
# not for the whole suite.

The short version

1. Use an image with Chromium already installed. `playwright install` is the slowest part of a cold browser-test job and it is pure waste on every run.
2. Clone the repository inside the sandbox at an explicit commit. Faster than uploading, and it pins what you tested.
3. Set `--workers` from memory, not cores. On a 4 GiB guest that is two or three for an app-heavy suite. Overcommitting produces flakiness that looks like your tests.
4. Shard with `--shard=i/n`, one shard per sandbox, and report with `blob` so the shards merge into one report afterwards.
5. Give each shard its own test data. Sharding exposes data races you already had.
6. Use `--trace=retain-on-failure`, and download artifacts in a `finally` block — the failing run is the one whose trace you need.
7. For interactive debugging, serve the report from the sandbox and open its preview URL instead of downloading archives.