all posts

What Is a Headless Browser? How It Works and What It Costs to Run

Ajay Kumar··12 min read

A headless browser is a normal web browser running without a visible window. Same rendering engine, same JavaScript engine, same networking stack, same cookie jar — just no chrome around the page and no pixels on a screen. You drive it from code instead of with a mouse, and you ask it for the things you would otherwise look at: a screenshot, a PDF, the text of a page after its JavaScript finished running, the result of clicking a button.

That is the whole definition. The rest of this post is the part people get wrong, which is not the definition but the consequences: what is actually running when you launch one, which of the three or four things you installed is the browser, and why the process that felt free on your laptop turns into your largest infrastructure line item the moment you run thirty of them.

I run a compute platform where a decent share of the workloads are browsers, so I have a horse in this race and I will say plainly at the end where our answer is the wrong one. But most of this post is vendor-neutral, because the failure modes are the same whether you run Chromium on a laptop, in a container, in a microVM, or in somebody's hosted browser API.

The correction most people need: it is not a different browser

The single most common misconception is that "headless Chrome" is a stripped-down variant with its own quirks — a cousin of the real browser that renders slightly differently and can be blamed when a test fails. That belief was once partly true, and it is decreasingly true now, and understanding why is worth five minutes.

Chrome originally shipped headless mode as a separate implementation. It reused the rendering engine but had its own browser layer, and it deliberately did not implement large parts of what a real Chrome does: extensions, the permissions UI, printing paths, some of the plumbing around navigation and downloads. So if your page depended on any of that, headless genuinely behaved differently, and a generation of blog posts and Stack Overflow answers grew up around "headless Chrome behaves differently" as a fact of life.

Chrome later shipped a new headless mode that takes the opposite approach: it is the real browser binary, with the real browser layer, simply not painting to a screen. Over subsequent releases this became the default behaviour of the headless flag, and the old implementation was split out into a separate, smaller binary for people who specifically want the lightweight thing. I am being deliberately vague about which release did what, because the version numbers move and you should check the current documentation for the browser you are pinning rather than trust a blog post's snapshot of them.

The practical consequence: if you are on a current browser and a current automation library, the honest default assumption is that headless renders the same as headed. When a test passes headed and fails headless, the cause is much more often the environment — missing fonts, a different timezone, a smaller viewport, no GPU, a container with 64 MB of shared memory — than the headless-ness itself. The folklore outlived the bug.

A useful reframe: headless is not a mode of the browser so much as the absence of a windowing system. Everything the browser does above the compositor is unchanged. Everything about the machine the browser is running on has changed, and that is where your bugs live.

There is a second, older way to run a browser without a screen that still comes up and is worth naming so you do not confuse the two. Instead of asking the browser not to open a window, you can give it a fake screen: a virtual framebuffer such as Xvfb, on which a fully headed browser opens a fully real window that simply nobody is looking at. That approach is heavier — you are running an X server — and it is mostly legacy, but it is not obsolete. If you need something headless mode still does not support in your version, running headed on a virtual display is the escape hatch. Our own browser template ships Xvfb for exactly that reason.

What is actually running when you launch one

Launching a headless Chromium does not start a process. It starts a small operating system's worth of them.

Chromium is multi-process by design, and the design is a security design, not a performance one. There is a browser process that owns the window (or, headless, the absence of one), the profile, and the top-level coordination. There is a renderer process per site instance — not per tab, per site instance, because site isolation puts cross-origin iframes into their own renderers. There is a GPU process, even with no GPU, because that is where compositing and rasterization live. There is a network service process and a storage service process. There are utility processes that come and go for things like audio, or parsing a font, or decoding an image format.

So the mental model "a browser is a process I can count" is wrong before you have loaded a page. Run 'ps' after a single launch and you will see six to ten processes attributable to one browser. That matters for three reasons: memory accounting has to sum a process tree, killing the browser means killing a tree and not a PID, and the isolation properties you get depend on which of those processes an attacker lands in.

Headless also does not mean "does not render". The browser still parses HTML, builds the DOM, computes styles, does layout, and rasterizes — it simply throws the pixels into a buffer you can ask for instead of onto a display. On a machine with no GPU, that rasterization happens on the CPU through a software renderer, which is why a page full of canvas or WebGL costs you real CPU in a headless run and why your screenshots of GPU-heavy pages can look subtly different from your laptop's.

# Launch one "browser" and leave it running, then look at what exists.
$ node -e 'require("playwright").chromium.launch().then(()=>new Promise(()=>{}))' &
$ sleep 3

# How many processes is "one browser"?
$ ps -eo pid,rss,args | grep -c '[c]hrome'

# And what kind each one is — the --type= flag tells you:
$ ps -eo pid,rss,args | grep '[c]hrome' | grep -o '\-\-type=[a-z-]*' | sort | uniq -c
# typical shape, before any page has loaded:
#   (no --type)   the browser process
#   zygote        the fork template renderers are spawned from
#   gpu-process   compositing + rasterization, GPU or not
#   utility       network service, storage service, and friends
#   renderer      one per site instance once you navigate

# Sum the resident set across the tree — this is the number that matters:
$ ps -eo rss,args | grep '[c]hrome' | awk '{s+=$1} END {print s/1024 " MB"}'

Run that on your own machine before you size anything. The point is not any particular figure — it is that the sum across the tree is already well past what people assume a browser costs, and no page has loaded yet. (RSS across a shared-memory-heavy process tree also double-counts; there is a better measurement further down.)

The control layer people conflate with the browser

When somebody says "we use Playwright", they have named the steering wheel, not the car. There are three distinct layers here and keeping them apart resolves an enormous amount of confusion about versions, image sizes, and why a fix for one library does nothing for another.

  1. The browser engine — Chromium, Firefox's Gecko, WebKit. This is the thing that renders. It is hundreds of megabytes of compiled code and it is what you are actually paying for in RAM.
  2. The wire protocol — the remote-control interface the engine exposes. For Chromium this is the Chrome DevTools Protocol (CDP), the same protocol your DevTools window speaks: JSON messages over a WebSocket. The W3C WebDriver protocol is the other lineage, an HTTP request/response API originally from Selenium, and WebDriver BiDi is the newer bidirectional standard that is converging the two worlds.
  3. The client library — Puppeteer, Playwright, Selenium, and everything built on them. This is an ergonomic wrapper: it launches the engine, speaks the protocol, and gives you page.click() instead of hand-rolled JSON.

You can see the middle layer directly, and doing it once is the fastest way to stop thinking of the library as magic. Start a browser with a debugging port and talk to it with curl:

# Layer 1 + 2, no client library at all.
$ chromium --headless --remote-debugging-port=9222 --user-data-dir=/tmp/prof &

$ curl -s http://127.0.0.1:9222/json/version
{
  "Browser": "HeadlessChrome/...",
  "Protocol-Version": "1.3",
  "webSocketDebuggerUrl": "ws://127.0.0.1:9222/devtools/browser/6e1c..."
}

# Every page.goto() your library exposes is a JSON message over that socket:
#   {"id":1,"method":"Page.navigate","params":{"url":"https://example.com"}}

That WebSocket URL is the whole API. Puppeteer is, roughly, a well-designed typed client for it plus a browser downloader. Playwright is a client for it plus equivalent clients for patched Firefox and WebKit builds, plus a test runner, plus the auto-waiting semantics that make it pleasant. Selenium sits on the WebDriver lineage and drives the browser through a per-browser driver binary. None of them is a browser.

Why "npm install playwright" is not what makes the image huge

Here is the confusion that costs people the most disk. The npm package or the pip wheel for a browser automation library is small — it is JavaScript or Python. The browsers are a separate download, fetched by an explicit install step, into a cache directory outside your project.

# Small: the client library.
npm i -D playwright            # a few MB of JS

# Large: the actual engines, fetched into ~/.cache/ms-playwright (Linux)
npx playwright install chromium         # one engine
npx playwright install                  # chromium + firefox + webkit

# Larger still: the shared libraries those engines link against
npx playwright install --with-deps chromium   # runs apt-get for you; needs root

# Where it went:
du -sh ~/.cache/ms-playwright/*

Each engine is in the hundreds of megabytes unpacked. Install all three and add the system libraries they link against and you are into gigabytes before your application code exists. This is why the official browser images are large, why "just add Playwright to our slim Alpine image" turns into an afternoon, and why the first thing to do when your CI image is bloated is check whether you installed three engines to test one.

Install exactly the engines you use, and pin them. The library version and the browser build it expects travel together — a library upgrade that silently pulls a new engine build is a classic source of "nothing changed and the screenshots all shifted by a pixel".

A related trap: the install step writes to a cache in the home directory of whoever ran it. Build the image as root, run the container as a non-root user, and the browsers are somewhere that user cannot read. Either set the browsers path explicitly to a shared location, or do the install as the user that will run it.

What people actually use one for

Four families of workload, and they stress the browser in genuinely different ways.

Screenshots and PDF rendering

The oldest use. You have HTML — an invoice, a report, a social preview card, a chart — and you need a PNG or a PDF that looks like what a human would see. A browser is the highest-fidelity HTML renderer that exists, so people use it as one, headlessly, on a server.

This workload is bursty and short: launch, render one page, produce one artifact, exit. It is extremely sensitive to the environment issues later in this post, because the whole output is a picture and any missing font or wrong timezone is visible in the deliverable. It also has the sharpest security profile if the HTML is user-supplied, which is its own subject and one I have written about separately.

End-to-end tests

Drive your own application the way a user would and assert on what happens. This is the highest-volume browser workload in most companies, and it is the one where concurrency bites first, because a test suite's whole point is to run a hundred things at once and finish before the coffee does.

Tests are also where the headed-versus-headless folklore is strongest, because a test that fails only in CI is exactly the shape of bug that invites a superstitious explanation. Nine times out of ten it is the environment, and the way to prove it is to reproduce the environment rather than the test.

Scraping pages that are JavaScript-rendered

If a page ships an empty div and fills it from an API call, an HTTP client plus an HTML parser sees the empty div. A browser sees the filled one, because it actually runs the JavaScript. That is the entire reason browsers show up in data pipelines that would otherwise be a hundred lines of requests and BeautifulSoup.

It is also worth saying: reach for this second, not first. Check the network tab. Very often the page is calling a JSON endpoint you can call directly, and a browser is three orders of magnitude more expensive than an HTTP GET. Use the browser when the rendering genuinely is the value, not as a default.

LLM agents driving a page

The newest family and the one growing fastest. An agent is handed a browser as a tool: it gets the accessibility tree or a screenshot as observation, and emits clicks, types, and navigations as actions. Sometimes it is filling a form in a system with no API. Sometimes it is researching across arbitrary sites.

This one is different in kind from the other three, and the difference is trust. A test suite visits your own application. A screenshot service visits URLs you chose. An agent visits whatever the model decided to visit, possibly because a page it already read told it to. The browser session becomes a place where untrusted content meets an untrusted decision-maker, and the isolation question stops being theoretical. It is the reason I think a per-session hardware boundary is the right default for agent browsing, and I will come back to it.

The smallest useful example

Here is a complete screenshot script with nothing clever in it. Python first:

# pip install playwright && playwright install chromium
from playwright.sync_api import sync_playwright

with sync_playwright() as pw:
    browser = pw.chromium.launch()           # headless is the default
    ctx = browser.new_context(
        viewport={"width": 1280, "height": 800},
        device_scale_factor=2,               # retina-ish output
    )
    page = ctx.new_page()
    page.goto("https://example.com", wait_until="networkidle")
    page.screenshot(path="shot.png", full_page=True)
    browser.close()                          # kills the whole process tree

And the Node equivalent, because half the ecosystem lives there:

// npm i playwright && npx playwright install chromium
const { chromium } = require("playwright");

(async () => {
  const browser = await chromium.launch();
  const ctx = await browser.newContext({
    viewport: { width: 1280, height: 800 },
    deviceScaleFactor: 2,
  });
  const page = await ctx.newPage();
  await page.goto("https://example.com", { waitUntil: "networkidle" });
  await page.screenshot({ path: "shot.png", fullPage: true });
  await browser.close();
})();

Two things in there are load-bearing and easy to skip. The explicit viewport, because the default is small and a page that is responsive will render its mobile layout if you forget. And browser.close() in a path that always runs, because a browser you forget to close is a browser that is still resident when the next one starts. More on that below, since it is one of the two ways people actually run out of memory.

PDF is the same shape, with one caveat worth knowing: PDF generation is a Chromium capability. The API exists on Chromium pages and not on the WebKit and Firefox ones, so a cross-browser test suite and a PDF service have different engine requirements.

page.goto("file:///workspace/invoice.html", wait_until="load")
page.pdf(
    path="invoice.pdf",
    format="A4",
    print_background=True,      # off by default; your CSS backgrounds vanish without it
    margin={"top": "12mm", "bottom": "12mm", "left": "12mm", "right": "12mm"},
)

The resource reality: a browser is not a lightweight process

This is the part that surprises everyone, including people who have shipped browser automation before, because the surprise is not that browsers use memory — everyone knows that — it is where the ceiling actually sits and how badly it behaves when you hit it.

Start with the shape of the number rather than the number. A headless Chromium with one simple page open costs a few hundred megabytes across its process tree. A heavy single-page application — a large JS bundle, a component tree, a client-side router — costs noticeably more. A page with several cross-origin iframes costs more again, because site isolation gives each of those origins its own renderer process. Video, canvas, and WebGL can push a single tab past a gigabyte on their own.

I am giving you ranges and not a precise figure on purpose, and I would be suspicious of anyone who gives you a precise one. The number depends on your pages, your browser build, your flags, and how long the process has been alive. What you need is not my number, it is a procedure for getting yours.

How to measure it properly

The naive measurement — resident set of the browser process — is wrong by a factor of several, because it misses the renderers, the GPU process, and the services. The second-naive measurement — sum of RSS across the tree — double-counts shared memory, and browsers share a lot of it. For capacity planning, the measurement that actually predicts when you fall over is the one the kernel uses to decide to kill you: the peak usage of the cgroup the whole tree lives in.

# Run the browser workload inside its own cgroup and read the peak.
# (cgroup v2; on a systemd host this is what a scope or a service already gives you.)
sudo systemd-run --scope --unit=browser-probe \
  -p MemoryAccounting=yes \
  node scrape.js

# While it runs, or after, from the same unit's cgroup:
cat /sys/fs/cgroup/system.slice/browser-probe.scope/memory.peak
cat /sys/fs/cgroup/system.slice/browser-probe.scope/memory.current

# In a container, the same numbers without systemd:
cat /sys/fs/cgroup/memory.peak

Run that against your real pages, at your intended concurrency, for as long as a real shift lasts. Then ask the one question that decides your architecture: does the line plateau, or does it climb? A plateau means your safe concurrency is a constant you can divide RAM by. A climb means your safe concurrency is a function of uptime, and you need a recycling policy before it reaches your ceiling. Which of those two you are in is most of the capacity work; my colleague-in-spirit on this blog has the longer version in the post on browser automation concurrency limits, including the pool-versus-fresh-machine trade.

Why the ceiling is RAM and not CPU

People size browser fleets by core count and get it wrong by a wide margin in both directions. Browsers spend most of their wall-clock time waiting: on DNS, on TLS, on the server, on a lazily-loaded image, on a timeout somebody set to three seconds. Averaged over a job, a renderer uses a fraction of a core. Ten concurrent browsers on four cores is usually fine on CPU.

Memory does not work like that. Memory is occupied for the whole duration of the session, whether the page is computing or waiting on the network. It does not average down. So the arithmetic is: usable RAM, minus what the OS and your own code need, divided by your measured per-session peak. That is your concurrency. CPU shows up as a secondary constraint on CPU-heavy pages — heavy canvas, video decode, giant DOM reflows — and as a first-order one only if you are doing software rasterization of graphics-heavy content at volume.

The failure mode past the memory ceiling is the expensive part. You do not get a clean "out of memory" from your automation library. You get the kernel's OOM killer choosing a renderer, and your library reporting a target that closed, or a navigation that timed out, or an element that never appeared. Every one of those looks like a flaky test or a broken site. Teams spend weeks chasing flakiness that was a capacity problem the whole time. The tell is that the failure rate tracks concurrency rather than any particular test.

The environment a browser assumes it has

A browser was written for a desktop. It assumes a windowing system, a font server, a user's locale, a generous /dev/shm, and a parent process that reaps its children. Servers and containers provide some of those and not others, and the gaps produce a specific catalogue of bugs that look like anything except what they are.

/dev/shm, or: the crash that has nothing to do with your code

Chromium moves rendered frames between its processes through shared memory. On a normal Linux machine /dev/shm is a tmpfs sized from real RAM, typically half of it, and this is a non-issue. The default in a container is 64 MB, which a moderately complex page will exhaust, at which point the browser dies with an error that mentions nothing about shared memory.

# The fix, in order of preference.

# 1. Give it real shared memory.
docker run --shm-size=1g my-scraper

# 2. Or mount a host tmpfs over it.
docker run --mount type=tmpfs,destination=/dev/shm,tmpfs-size=1g my-scraper

# 3. Only if you cannot do either: make Chromium use temp files instead.
#    Costs performance. This flag is popular purely because option 1 is
#    often not available to the person debugging at 2am.
chromium --disable-dev-shm-usage

On a VM this problem does not exist, because /dev/shm is sized from the machine's actual memory. That is a small instance of a general pattern worth internalising: a large fraction of browser automation folklore is workarounds for container-specific constraints, carried forward as universal truths into environments that never had the constraint.

Fonts and emoji, or: why your screenshot is full of boxes

A slim server image has approximately no fonts. The browser does not fail — it falls back, and keeps falling back, until it renders the replacement glyph, which is the little box people call tofu. Your text is there in the DOM, your tests may even pass, and your screenshot is unusable.

Three separate things go missing and people usually fix only the first. Latin text needs a base font family. Non-Latin text — Chinese, Japanese, Korean, Arabic, Devanagari, Thai — needs its own coverage and a slim image has none of it, so an internationalised product renders as boxes for exactly the customers you care about most. And emoji need a colour emoji font specifically, or they render as monochrome outlines or not at all.

# Debian/Ubuntu. Install what you actually render, not the world:
RUN apt-get update && apt-get install -y --no-install-recommends \
      fonts-liberation \
      fonts-dejavu-core \
      fonts-noto-core \
      fonts-noto-color-emoji \
      fonts-noto-cjk \
      fontconfig \
 && fc-cache -f \
 && rm -rf /var/lib/apt/lists/*

# Verify inside the image, before you find out from a customer:
#   fc-list | wc -l
#   fc-match "sans-serif"
#   fc-match "emoji"

I will hold myself to the same standard: our own browser template installs the shared libraries Chromium links against and does not install a full font set. That is fine for scraping text out of pages and wrong for a screenshot service, and if you are building the latter on it, adding fonts is your first commit. The general rule is that font coverage is a property of your image and nobody else will get it right for you.

A related detail for anyone doing visual regression: fonts must be pinned, not merely present. A base image update that bumps a font package will shift text rendering by a subpixel across your entire baseline set and produce a diff on every single screenshot. Pin the font packages the way you pin the browser.

Timezone and locale change what gets rendered

Containers default to UTC. Your laptop does not. Any page that formats a date, a time, a currency, or a number in the client renders differently in the two places, and if you are diffing screenshots or asserting on text, that difference is a failing test that has nothing to do with your change.

There are two levels to set this and you want the second one. The process level, via the TZ environment variable, applies to everything in the container. The context level, via your automation library, applies per browser context, which means one browser can render a page as Tokyo and another as Berlin without you running two containers.

ctx = browser.new_context(
    timezone_id="America/New_York",   # what Date and Intl see
    locale="en-US",                   # Accept-Language + Intl formatting
    color_scheme="light",             # pin it; a base image change can flip this
    reduced_motion="reduce",          # kills animation flake in screenshots
    viewport={"width": 1280, "height": 800},
)
# The process-level version, for anything not going through a context.
ENV TZ=UTC
RUN apt-get update && apt-get install -y --no-install-recommends tzdata \
 && ln -fs /usr/share/zoneinfo/$TZ /etc/localtime \
 && dpkg-reconfigure -f noninteractive tzdata

Pick one timezone for your fleet, set it explicitly rather than inheriting it, and set the interesting per-test values at the context level. Explicit-and-boring beats implicit-and-correct-on-your-machine.

No display, no GPU, no sound

Headless mode handles the missing display for you. It does not conjure a GPU. On a server without one, compositing and rasterization fall back to a software renderer, which is correct but slower and occasionally pixel-different from a hardware path. For most work this is invisible. For WebGL-heavy pages it is the difference between a screenshot in two seconds and one in thirty, and it is worth measuring rather than assuming.

You may also see advice to pass a flag disabling the GPU. On modern Linux headless this is usually unnecessary and sometimes counterproductive, and it is another piece of folklore whose original reason (a Windows-specific bug, years ago) has long since been fixed. Try without it first.

--no-sandbox: what you are actually turning off

Every guide to running Chromium in Docker eventually says: add --no-sandbox. It works. It is also the single worst-understood line in browser automation, and it is worth understanding precisely, because the honest position is not "never do this" — it is "know exactly what you traded and get it back somewhere else".

Chrome has its own sandbox, and it is good

Chromium's whole security architecture rests on the assumption that the renderer process is compromised. Renderers parse the most hostile input on the internet — arbitrary HTML, CSS, images, fonts, JavaScript — and renderer exploits are a real, regularly-patched category of bug. The design's answer is that a compromised renderer should be a useless prize: it runs in a heavily restricted sandbox with almost no ability to touch the filesystem, the network directly, or other processes.

On Linux that sandbox is built in two layers. The first uses Linux namespaces — in particular user namespaces — to strip the renderer of its view of the system: no filesystem it should not see, no PIDs it should not see, no network namespace of its own to abuse. The second is a seccomp-BPF filter that shrinks the syscall interface the renderer can reach down to a small allowlist, so even if the code inside is fully attacker-controlled, the surface it can attack in the kernel is tiny.

Why containers break it

Both layers need kernel features that container runtimes commonly restrict, and for good reasons of their own. Creating a nested user namespace is exactly the primitive that a container escape wants, so runtimes and hardened kernels often block or restrict it. Installing a seccomp filter requires a syscall that a container's own seccomp profile may not permit. Put a browser inside a container with default settings on a hardened host and Chromium's attempt to build its sandbox fails, and Chromium — reasonably — refuses to start rather than run unprotected without telling you.

At which point the internet says: pass --no-sandbox. And the browser starts, and the test suite goes green, and nobody thinks about it again.

What --no-sandbox actually does: it runs the renderer with the same privileges as your automation process. A renderer exploit — from a hostile page, an ad, an image, a font — now executes with your process's access to the filesystem, the network, your environment variables, and any credentials mounted into the container. The only boundary left between a hostile web page and the host is the container's shared-kernel boundary, which is the one you were relying on the browser sandbox to back up.

What to do instead

The risk is a function of what you visit. Be honest about which case you are in.

  • You visit only your own application, in CI, on a network with no secrets reachable — the practical risk of --no-sandbox is low. Say so in a comment next to the flag, so the next person knows it was a decision and not a copy-paste, and revisit it if the workload ever starts visiting third-party URLs.
  • You visit arbitrary or user-supplied URLs — screenshot services, scrapers, agents browsing the open web — and --no-sandbox is not an acceptable answer. You have deliberately removed the mitigation designed for exactly your threat model.

The fixes, roughly in order of how much they cost you:

  1. Give the container the kernel features the browser needs, rather than telling the browser to give up. Applying Chromium's own published seccomp profile to the container, or permitting unprivileged user namespaces where your kernel and runtime allow it, lets the browser build its sandbox normally. This is fiddly, host-dependent, and the details change with runtime versions — verify against your own runtime's current documentation.
  2. Add a second boundary that does not depend on the first. If the browser genuinely cannot sandbox itself, then something else must be the wall: a dedicated machine per job, a VM, a network with nothing worth reaching. --no-sandbox inside a throwaway VM with locked-down egress is a very different risk than --no-sandbox in a pod that can reach your cluster's metadata endpoint and its service account token.
  3. Run it on a real kernel of its own, and let the browser sandbox work as designed. This is the microVM answer and I am biased, so I will make the argument properly in a moment rather than smuggle it in here.

The thing I would push back on hardest is the framing that this is a choice between "secure" and "works". It is a choice about where the boundary lives. Turning off the browser's sandbox is defensible when there is a stronger boundary underneath it. It is indefensible as the only change you make to get a scraper of the open internet running in production.

Zombie processes and leaked browsers

Two related resource leaks, both of which show up as "the box slowly dies over a day" rather than as an error, which is why they take so long to find.

Zombies: nobody is reaping the children

On Linux, when a process exits, its parent must call wait() to collect the exit status. Until it does, the dead process stays in the table as a zombie: no memory, but a PID and a table entry. Normally init does this for orphans. In a container, PID 1 is your application — a Node script, a Python entrypoint — and application runtimes are not written to reap arbitrary orphans.

A browser spawns and reaps a lot of children over a session: renderers per navigation, utility processes for a font here and an image decode there. Every one whose parent died before it produces an orphan that nobody collects. Run for hours and the process table fills, and then nothing on the box can fork, and your error message is about being unable to allocate memory when there is memory everywhere.

# Look for them before they look for you.
ps -eo pid,ppid,stat,comm | awk '$3 ~ /^Z/'
ps -eo stat | grep -c '^Z'

# The fix is one flag: a real init as PID 1 that reaps orphans.
docker run --init my-scraper           # tini, bundled with Docker

# In Kubernetes there is no --init; use shareProcessNamespace, a tini
# entrypoint in the image, or an init system in the container.

Leaked browsers: close() in a finally, and a backstop above it

The bigger leak is whole browsers. Your job throws before browser.close(); the exception propagates; the job is retried; the browser is still running with its full process tree resident. Do that forty times over an afternoon and the machine is gone.

Language-level cleanup is necessary and insufficient. Necessary: always close in a finally or a context manager, never on the happy path only.

from contextlib import closing

# Good: closes on exception, on timeout, on assertion failure.
with sync_playwright() as pw:
    browser = pw.chromium.launch()
    try:
        page = browser.new_page()
        page.goto(url, timeout=30_000)
        page.screenshot(path=out)
    finally:
        browser.close()

Insufficient, because none of that runs if the process is SIGKILLed, if the OOM killer takes your Python and leaves the browser, or if the container is evicted mid-job. Those are exactly the situations where a leak is most likely, and language-level cleanup is structurally unable to help. You need a backstop outside the process:

  • A per-job timeout enforced by something that outlives the job — a supervisor, a queue's visibility timeout, a CI step timeout.
  • A janitor that kills browser process trees older than N minutes, if you are running a pool on a long-lived box.
  • Or: make the unit of cleanup the machine rather than the process. If the job runs in a VM with a TTL, a leaked browser is not a leak — it is deleted along with everything else when the VM goes, and no cleanup code had to run correctly.

That last one is the argument for disposable environments in a nutshell, and it generalises far past browsers: cleanup logic that has to run is cleanup logic that will eventually not run. Cleanup that is a consequence of the environment disappearing cannot be skipped.

What it costs, in money rather than megabytes

Put the pieces together and the cost model falls out. A browser session occupies memory for its whole duration regardless of what it is doing. Your machine holds a fixed number of them. So the unit you are buying is RAM-seconds, and your bill is sessions times duration times footprint, divided by however well you pack them.

Which means there are exactly four levers, and it is worth knowing which one you are pulling:

  1. Fewer sessions. The cheapest browser is the one you did not launch. Check whether the page has a JSON endpoint. Check whether a cached artifact would do. This lever is usually the largest and almost always the least explored.
  2. Shorter sessions. Do not wait for networkidle on a page with a polling websocket; it will never idle and you will pay for the timeout. Wait for the specific thing you need. Block image, font, and media requests you are not going to look at — on a scraping workload that alone can halve wall-clock time and memory both.
  3. Smaller footprint per session. Contexts instead of browsers for parallel work within one trust boundary — a context costs tens of megabytes where a browser costs hundreds. Be clear-eyed that a context is not a security boundary: contexts share a process tree and an OS user, so a renderer exploit crosses between them freely. Contexts are a density tool, not an isolation tool.
  4. Less idle. This is the one that quietly dominates real bills. A warm pool sized for peak is paying for peak twenty-four hours a day. If your traffic is bursty — and browser traffic almost always is, because it is tied to business hours or to CI runs or to a cron — the gap between peak-sized capacity and actual usage is the majority of the spend.

The fourth lever is where the architecture choice bites. A pool of long-lived browsers gives you the fastest job start and the worst idle economics, plus a recycling policy you have to maintain because memory climbs over a pool's lifetime. A fresh machine per job gives you perfect isolation, zero idle, and no recycling policy — and traditionally it lost on latency, because booting a machine took seconds to minutes and nobody will wait that long for a screenshot.

Snapshot-restore is what changed that arithmetic, and it is the reason I think per-job machines are now the default rather than the luxury option.

Where a microVM per browser session fits

Now the biased part, flagged as such. PandaStack runs every sandbox as its own Firecracker microVM, and for browser workloads that buys three specific things. Not vibes — three mechanisms.

A real kernel, so the browser's own sandbox works

The container problem earlier in this post is a shared-kernel problem: the browser wants to create user namespaces and install seccomp filters, and the runtime restricts exactly those because on a shared kernel they are dangerous primitives. A microVM does not share a kernel. There is a real guest kernel in there, the guest is the only tenant, and the browser can build its sandbox the way it was designed to — no --no-sandbox, no seccomp profile archaeology, no arguing with your platform team about capabilities.

And the two boundaries then compose properly. A renderer exploit has to break out of Chromium's sandbox, and then out of the guest kernel, and then out of the hypervisor, to reach anything of yours. That is the layering the browser's designers assumed and that a single shared kernel collapses into one layer. If you want the longer version of the kernel-boundary argument, it is in the microVM explainer.

A hard memory boundary per session

The noisy-neighbour failure I described earlier — one heavy page triggering an OOM kill that lands on somebody else's renderer, surfacing as a flaky test in an unrelated job — is a consequence of sharing one memory pool. A microVM's guest RAM is fixed at its size. A session that leaks memory hits its own ceiling, and the blast radius is that session. Everything else on the host is unaffected, which turns an unattributable fleet-wide flake into a single failed job with an obvious cause.

The honest caveat: that ceiling is fixed at snapshot-bake time, not per request. Our browser template is baked at 4 GiB of guest RAM and 8 vCPU, and a create cannot change it, because Firecracker cannot resize a guest's memory at snapshot restore. If your pages need a different size, that is a template you bake, not a parameter you pass. I would rather tell you that than let you discover it.

A snapshot with the browser already warm

The latency objection to per-job machines is boot time. Snapshot-restore removes it: instead of cold-booting, every create restores a baked snapshot of an already-booted machine, which lands at roughly 179ms p50 for us. That is fast enough that a fresh VM per job stops being a thing you optimise around.

The better trick is what you put in the snapshot. Because the snapshot captures guest memory and device state, you can snapshot a machine on which Chromium is already launched and listening on a debug port. Restore that, and you have a running browser without paying the launch cost at all — the process tree is already there, the engine is already initialised.

from pandastack import Sandbox

# --- One time: bake a snapshot with the browser already running. ---
warm = Sandbox.create(template="browser", ttl_seconds=3600)
warm.exec(
    "setsid chromium --headless --no-first-run "
    "--remote-debugging-port=9222 --user-data-dir=/tmp/prof "
    "> /var/log/chromium.log 2>&1 < /dev/null &"
)
warm.exec("until curl -sf http://127.0.0.1:9222/json/version >/dev/null; do sleep 0.2; done")
snap_id = warm.snapshot()      # captures guest memory + device state + disk
warm.delete()

# --- Per job: restore it. The browser is already up inside. ---
job = Sandbox.create(from_snapshot=snap_id, ttl_seconds=300)
print(job.exec("curl -s http://127.0.0.1:9222/json/version").stdout)
job.delete()
Two honest caveats on that pattern. First, a restored browser's pre-existing network connections are stale — the guest came back with sockets that the other end forgot about years ago in wall-clock terms. Snapshot after launch and before navigation, and let each job do its own navigating. Second, fork() on PandaStack today copies on-disk state, not memory: a forked child boots from the parent's disk and does not inherit a running process tree. For a warm-in-RAM browser you want a snapshot restore, not a fork. Fork is the right tool for branching disk state — a logged-in profile directory, a populated cache — and I would rather correct that here than have you build on a wrong mental model.

Add scale-to-zero on top and the fourth cost lever from the previous section goes to roughly nothing: there is no warm pool to pay for between bursts, because the warm state lives in a snapshot on disk rather than in idle machines holding RAM.

Running one inside a sandbox, end to end

Here is the whole thing as a working example: create an isolated microVM with the browser stack preinstalled, drop a script in, run it, pull the PNG back out, throw the machine away.

# pip install pandastack
from pandastack import Sandbox

SCRIPT = '''
from playwright.sync_api import sync_playwright
import sys

url = sys.argv[1]

with sync_playwright() as pw:
    browser = pw.chromium.launch()          # real kernel: no --no-sandbox needed
    try:
        ctx = browser.new_context(
            viewport={"width": 1440, "height": 900},
            device_scale_factor=2,
            timezone_id="UTC",
            locale="en-US",
            reduced_motion="reduce",
        )
        page = ctx.new_page()
        # Don't pay for pixels you're not going to look at.
        page.route("**/*.{woff,woff2,mp4,webm}", lambda r: r.abort())
        page.goto(url, wait_until="domcontentloaded", timeout=30_000)
        page.wait_for_load_state("networkidle", timeout=10_000)
        page.screenshot(path="/workspace/shot.png", full_page=True)
        print("ok", page.title())
    finally:
        browser.close()
'''

# ttl_seconds is the backstop: even if this process dies, the VM (and any
# browser leaked inside it) is reaped. Cleanup you don't have to execute.
with Sandbox.create(template="browser", ttl_seconds=300) as sbx:
    sbx.filesystem.write("/workspace/shot.py", SCRIPT)

    res = sbx.exec("python3 /workspace/shot.py https://example.com", timeout_seconds=120)
    print(res.stdout, res.stderr, res.exit_code)

    sbx.filesystem.download("/workspace/shot.png", "shot.png")

# VM gone. Browser gone. Zombies gone. Nothing to reap, nothing to recycle.

The TypeScript SDK is the same shape if that is your stack:

import { Sandbox } from "@pandastack/sdk";

const sbx = await Sandbox.create({ template: "browser", ttlSeconds: 300 });
try {
  await sbx.filesystem.write("/workspace/shot.js", SCRIPT);
  const res = await sbx.exec("node /workspace/shot.js https://example.com");
  console.log(res.stdout, res.exitCode);
  await sbx.filesystem.download("/workspace/shot.png", "./shot.png");
} finally {
  await sbx.delete();
}

Two details in the Python version are the point of the whole post. There is no --no-sandbox, because the guest kernel is real and the browser can protect itself. And the cleanup that matters is the TTL, not the close() — the close() is good hygiene, the TTL is the thing that still works when the process is killed mid-job.

When you do not need any of this

The rule in this corpus is that if a post concludes "use us" for every reader it is worthless, so here is where I would not use us, and where I would not use a browser at all.

Do not use a browser when the page is not actually JavaScript-rendered. Open the network tab, find the JSON call, hit it directly. An HTTP request and a parser cost you a few megabytes and a few milliseconds where the browser costs hundreds of megabytes and seconds. This is the single most common over-engineering in this whole space and I have done it myself more than once.

Do not build a per-job VM architecture for a single nightly screenshot job. If you render one report at 3am, a container on a box you already own is completely fine. Give it --shm-size, --init, and fonts; take the small risk on --no-sandbox if it only ever visits your own HTML; go to bed. The isolation and elasticity arguments in this post are arguments about scale, untrusted input, or both, and one predictable job against trusted content has neither.

Do not move off a browser pool that is working. If you have a tuned pool with a recycling policy, steady traffic that keeps it busy, and no untrusted URLs, the per-job model's main wins — zero idle and per-job isolation — are wins you are not currently missing. Steady traffic is precisely the case where a pool's idle economics are fine.

And do not reach for a platform when the thing you need is a hosted browser API. If your product is "give an agent a browser" and you want somebody else to own session management, proxies, and captcha-adjacent problems, there is a whole category of hosted browser services aimed exactly at that, and they will get you further faster than assembling one on generic compute. What we sell is a general-purpose isolated machine that a browser happens to run well inside; that is the right shape when the browser is one step in a larger pipeline that also compiles code, runs a database, or executes whatever an agent decided to run. Verify pricing and limits against current documentation for anything in that category, including ours — it moves quickly.

The short version

  • A headless browser is the same browser engine with no window. Modern headless is the real browser binary, so "headless renders differently" is folklore more often than fact — suspect the environment first.
  • You are running a process tree, not a process: browser, renderers per site instance, GPU, network and storage services. Count and kill trees, not PIDs.
  • The browser, the protocol (CDP or WebDriver), and the client library (Puppeteer, Playwright, Selenium) are three separate layers. Installing the library is small; installing the engines is what makes your image gigabytes.
  • Memory is the ceiling, not CPU. Measure your own peak with cgroup accounting against your real pages, and find out whether it plateaus or climbs before you pick a pool size.
  • Past the ceiling you get OOM kills that surface as flaky tests. If failure rate tracks concurrency rather than a specific test, it is capacity.
  • The environment gaps are: /dev/shm too small in containers, no fonts (Latin, CJK, and emoji are three separate problems), UTC instead of your timezone, no GPU.
  • --no-sandbox turns off Chromium's own renderer sandbox, which is exactly the mitigation for hostile pages. Acceptable against your own content behind another boundary; not acceptable for the open web with nothing underneath it.
  • Leaks come from unclosed browsers and unreaped children. Language-level cleanup does not run when you are SIGKILLed. Make the environment disposable instead.
  • A microVM per session gives the browser a real kernel to sandbox itself in, a hard per-session memory ceiling, and a snapshot you can restore with the browser already warm. For one nightly job on a box you own, that is overkill and a container is fine.

Frequently asked questions

What is a headless browser?

A headless browser is a normal web browser running without a visible window — the same rendering engine, JavaScript engine, and networking stack, driven from code instead of by a mouse. It still parses HTML, runs JavaScript, computes layout, and rasterizes pages; it simply renders into a buffer you can request (as a screenshot or PDF) instead of onto a screen. Headless Chrome, headless Chromium, and headless Firefox are the same browsers you already use, minus the window.

Is headless Chrome a different browser from regular Chrome?

Not any more, in practice. Chrome originally shipped headless as a separate implementation that lacked parts of the real browser layer, which is where the "headless behaves differently" reputation came from. Chrome later shipped a new headless mode that is the real browser binary simply not painting to a screen, and that became the default. On current versions, when a page renders differently headless, the cause is usually the environment — missing fonts, a different timezone, a smaller viewport, no GPU, or too little shared memory — rather than headless mode itself.

How much memory does a headless browser use?

More than people expect, and you must sum a process tree rather than one process: a browser process, a renderer per site instance, a GPU process, and network and storage services. A simple page costs a few hundred megabytes across that tree; a heavy single-page app costs noticeably more; cross-origin iframes each add a renderer; and canvas, video, or WebGL pages can exceed a gigabyte alone. Measure your own workload with cgroup memory accounting (memory.peak) against your real pages, and check whether usage plateaus or climbs over a long session.

Is --no-sandbox safe for headless Chrome in Docker?

It removes Chromium's own renderer sandbox, which is the mitigation designed for exactly the case where a hostile web page compromises a renderer. With it off, a renderer exploit runs with the privileges of your automation process — its filesystem access, network access, environment variables, and mounted credentials. It is defensible when the browser only ever visits content you control and another boundary sits underneath it, and it is not defensible for scraping or agent browsing on the open web. Better options are to let the container create user namespaces and install seccomp filters so the browser can sandbox itself, or to run the browser on its own kernel in a VM or microVM.

What is the Chrome DevTools Protocol, and how does it relate to Puppeteer and Playwright?

The Chrome DevTools Protocol (CDP) is the remote-control interface a Chromium browser exposes — JSON messages over a WebSocket, the same protocol the DevTools window uses. Puppeteer and Playwright are client libraries that speak it for you; Playwright also drives patched Firefox and WebKit builds. Selenium comes from the other lineage, the W3C WebDriver HTTP protocol, and WebDriver BiDi is the newer bidirectional standard converging the two. The library is not the browser: installing Playwright or Puppeteer is a small package, and the separate browser install step is what downloads hundreds of megabytes per engine.

Why do headless browser screenshots show boxes instead of text or emoji?

Because the image has no fonts. Slim server images ship almost none, so the browser falls back until it renders the replacement glyph ("tofu"). Three separate gaps cause this: no base Latin family, no coverage for non-Latin scripts such as CJK, Arabic, or Devanagari, and no colour emoji font. Install the font packages you actually render, run fc-cache, and verify inside the image with fc-list and fc-match. For visual regression testing, pin the font packages too — a base image update that bumps a font will shift text rendering across your entire baseline.

Keep reading

Run code in a microVM in one API call.

49ms p50 cold start. Fork, snapshot, and scale to zero.

Start free
Written by Ajay Kumar, Founder, PandaStack.