Running a Plugin Marketplace Without Getting Owned

A plugin marketplace is a supply chain you invited into your product on purpose. Slack apps, Shopify apps, IDE extensions, an AI-tool directory — the pitch is the same and it's a good one: let the community extend your platform and you get a thousand features you never had to build. The catch is that every one of those features is code written by someone you've never met, running against your users' data, on your infrastructure, on your reputation. You are, whether the org chart says so or not, in the business of executing strangers' code. This post is about the one design decision that decides whether that business is survivable: what wall stands between a plugin and everything it shouldn't touch.

The threat: review passed v1.0, nobody re-reads v1.1

The comforting story is that a review process catches bad plugins at submission. It catches some. What it structurally cannot catch is the classic supply-chain move: ship a benign, genuinely useful v1.0, pass review, accumulate installs and trust, and then quietly push a malicious v1.1 into the same slot the user already granted permissions to. Nobody re-reviews the update with the same scrutiny as the first submission — the marketplace has thousands of updates a week and a review team of a dozen — so the malicious version rides in on the reputation the honest version earned. This isn't hypothetical; it's the dominant pattern in real-world extension and package attacks.

Enumerate what a plugin can reach for once it's inside, because the list is the threat model:

• Steal the tokens it was handed: a Slack app holds an OAuth token, a Shopify app holds an admin API scope, an IDE extension holds whatever's in your dotfiles. A malicious update's simplest, highest-value move is to exfiltrate the credential it was legitimately given and use it — or sell it — later, from somewhere else.
• Read the user data it processes: the plugin's whole job is to touch user data — that's why it exists. A malicious version doesn't need an exploit to read what you deliberately passed it; it needs a way to send that data somewhere it shouldn't go.
• Dependency confusion and typo-squatting: the plugin author didn't turn evil — their build did. A poisoned transitive dependency, a typo-squatted package name, or a compromised CI pulls malicious code into an otherwise-honest plugin without the author ever knowing.
• Pivot into your infrastructure: if the plugin runs where it can open network connections, it can scan your internal network, hit private admin services, or GET the cloud metadata endpoint at 169.254.169.254 and walk off with the host's IAM credentials. That's a marketplace plugin turning into a foothold in your cloud account.

Why static review and containers aren't enough

Static review is a filter, not a wall. It reduces the rate of obviously-bad submissions, and you should absolutely do it — but it's probabilistic, it lags updates, and it's defeated by any attacker willing to be patient or to obfuscate. Betting your users' data on catching malice by reading code is betting on being smarter than every future submitter, forever. That's not a bet; it's a hope with a dashboard.

So teams reach for the runtime wall and pick a container. For first-party code that's fine. For arbitrary third-party plugins it's the wrong boundary, and the reason is architectural rather than a matter of tuning. A container is Linux namespaces and cgroups around a process that still shares the host's single kernel with every other plugin on the machine. That shared kernel — hundreds of syscalls, plus ioctls, the filesystem layer, reachable drivers — is the attack surface a malicious plugin probes to escape. You can harden a long way (seccomp, dropped capabilities, user namespaces, gVisor or Kata in front) and you should, but you're narrowing a shared kernel's surface, not removing the sharing. A kernel-escape bug reachable from inside a plugin is a breach of every plugin and every tenant on that box.

Review decides which strangers you let in. Isolation decides what they can do once you're wrong about one. You will be wrong about one.

The categorically stronger boundary is to stop sharing the kernel: run each plugin invocation in a microVM, a guest with its own kernel behind a CPU-enforced virtualization boundary (Intel VT-x / AMD-V via KVM). A compromised plugin is trapped inside a guest that can't see the host kernel at all. This is exactly why AWS built Firecracker to run Lambda between mutually distrusting tenants instead of trusting containers to hold that line.

The invocation model: input in, run in a VM, output out, VM gone

The runtime contract for a plugin should be a black box you can throw away. Every invocation follows the same five steps, whether the plugin is enriching a Shopify order, formatting a Slack message, or running an AI tool call:

1. Route to isolation: pick or create a microVM scoped to this plugin (or this plugin-for-this-tenant). For ephemeral-per-invocation, that's a fresh VM restored from the plugin's baked snapshot; for a hot plugin, it's a persistent VM you keep warm and reset between calls.
2. Deliver the input: write the invocation payload — the order, the event, the tool arguments — into the guest as a plain file or on stdin. The plugin code was pre-loaded when the plugin was published, so the VM already contains exactly the version you snapshotted.
3. Run with a hard timeout: exec the plugin against the input. Every exec returns stdout, stderr, and an exit code, so you can distinguish success from a crash from a plugin that decided to mine crypto until the clock ran out.
4. Capture the output: read the plugin's response back out — the one file or stream you deliberately extract. That response, plus whatever outbound calls you explicitly allowed, is the only thing that escapes the VM.
5. Destroy or reset: for ephemeral, delete the VM; nothing survives the invocation. For persistent, restore it from the clean snapshot so the next call starts from a known-good state, never from whatever the last call left behind.

Because the VM is disposable, you never write defensive cleanup between invocations. If a plugin corrupts its filesystem, leaks a token into an environment variable, or plants a foothold, it does so inside a VM that's about to be deleted or rolled back to a snapshot that never saw the attack. The blast radius is one invocation of one plugin — and if you scope per-tenant, it's one invocation of one plugin for one customer.

Scoping capabilities: give each plugin only what it needs

Isolation is the wall; capability scoping is deciding how many doors that wall has. A plugin that reformats text needs no network and no secrets. A plugin that syncs orders to a customer's warehouse needs to reach exactly one endpoint and nothing else. The mistake is a one-size posture where every plugin gets the same broad grant 'to be safe' — that's the opposite of safe. Scope each plugin declaratively at publish time and enforce it at the VM boundary at runtime:

• Which secrets it receives: a plugin gets only the tokens its declared function requires, minted per invocation and short-lived. Don't hand a formatting plugin your admin API key because it was convenient.
• Which network it may touch: default-deny, then allowlist the specific destinations the plugin declared. A plugin that claimed it only talks to `api.stripe.com` can't suddenly open a socket to an attacker's collector.
• Which resources it can burn: fixed vCPU and RAM at the VM boundary, plus a hard exec timeout, so a runaway or malicious plugin saturates its own allocation and no more — the noisy-neighbor problem, solved by the hypervisor instead of by hope.
• How long it lives: a TTL backstop so a plugin that hangs kills its own VM, and its capabilities expire with it.

Egress allowlisting: stop exfiltration and SSRF at the wire

Isolation stops a plugin from reading its neighbors. Egress control stops it from phoning home with what you handed it. These are different problems and a marketplace needs both, because the whole point of most malicious plugins is exfiltration — the data or the token leaves the building. A plugin perfectly isolated from other tenants can still POST a customer's records to an attacker's server, or turn inward and hit your own network: the database that trusts anything on the VPC, the admin service on a private port, the metadata endpoint that hands out cloud credentials. That inward turn is server-side request forgery, and a third-party plugin with free network access is a gift-wrapped SSRF primitive.

The microVM boundary is the natural enforcement point because every sandbox gets its own network namespace with its own TAP device and NAT rules — PandaStack pre-allocates 16,384 /30 subnets per agent, one per sandbox, so a plugin's network is genuinely its own segment rather than a shared bridge. Set the posture there:

• Default-deny outbound: most plugins that only transform their input need no network at all. Give them none. The safest plugin is one that can't open a socket.
• Block the internal ranges and metadata endpoint: for plugins that do need egress, deny RFC1918 (10/8, 172.16/12, 192.168/16) and the link-local 169.254.0.0/16 — which covers the cloud metadata IP at 169.254.169.254 — so a plugin can never reach your database, control plane, or IAM credentials.
• Allowlist, never blocklist: permit the exact destinations the plugin declared and nothing else. Blocklists lose the moment an attacker finds a host you forgot; allowlists fail closed by construction.
• Rate-limit and cap: bound how much a single plugin can send, so a compromised one can't become an exfiltration firehose or a DDoS amplifier billed to you.
• Keep the clock and TTL: an egress-heavy plugin that's also slow is a resource-exhaustion attack. The hard timeout is part of your network defense, not just your CPU defense.

Invoking a plugin in an isolated VM

Here's the whole invocation with the Python SDK: create a plugin-scoped microVM from the plugin's versioned snapshot, deliver the input payload, exec the plugin with a hard timeout, read the single response back out, and destroy the VM. Set PANDASTACK_API_KEY in the environment first. The `metadata` tag attributes the VM to a plugin, version, and tenant for auditing, and `ttl_seconds` is a backstop so a hung plugin reaps its own VM.

import json
from pandastack import PandaStack

ps = PandaStack()  # reads PANDASTACK_API_KEY, base url https://api.pandastack.ai

def invoke_plugin(plugin_id: str, version: str, tenant_id: str,
                  payload: dict) -> dict:
    # 1. Fresh, hardware-isolated microVM restored from THIS plugin version's
    #    baked snapshot (~179ms p50), so a VM-per-invocation is cheap. The
    #    metadata tags pin the VM to a plugin/version/tenant for the audit log.
    sb = ps.sandboxes.create(
        template=f"plugin-{plugin_id}-{version}",  # versioned snapshot
        ttl_seconds=45,                             # hard cap: self-destructs
        metadata={
            "plugin": plugin_id,
            "version": version,
            "tenant": tenant_id,
            "kind": "marketplace-plugin",
        },
    )
    try:
        # 2. Deliver the invocation input. Nothing about this call touches
        #    another plugin's VM, or another tenant's.
        sb.filesystem.write("/in.json", json.dumps(payload))

        # 3. Exec the untrusted plugin with a hard timeout. It reads /in.json,
        #    writes /out.json. A plugin that loops forever is killed at 20s.
        result = sb.exec(
            "node /plugin/entry.js < /in.json > /out.json",
            timeout_seconds=20,
        )

        # 4. Capture the response. This is the ONLY thing that escapes the VM —
        #    the token it was handed and the data it saw die with the guest.
        if result.exit_code != 0:
            return {"ok": False, "stderr": result.stderr[-2000:]}
        return {"ok": True, "result": json.loads(sb.filesystem.read("/out.json"))}

    finally:
        # 5. Destroy. No state, no token, no foothold survives the call.
        sb.delete()

The two safety rails aren't decoration. The `timeout_seconds` on `exec` is a circuit breaker for a plugin that loops forever; the `ttl_seconds` on create is a backstop so a VM you forget to reap reaps itself. Untrusted code gets a wall, a clock, and a hard cap — then it gets thrown away. If a plugin's function is long-running and you'd rather stream its output live than block, the SDK exposes a streaming exec that emits stdout, stderr, and exit events.

Versioned snapshots: fast installs, and a clean line between v1.0 and v1.1

The usual objection to a VM per invocation is cost, and for a cold-booted conventional VM the objection is correct — VMs boot in tens of seconds and hold gigabytes. A microVM snapshot restore is a different operation. A cold boot does real work: kernel init, userspace, runtime load. A restore skips all of it — Firecracker maps a frozen, already-booted VM's memory and device state back in and resumes mid-instruction. It's closer to unpausing than to starting a computer.

On PandaStack that restore-based create runs at a p50 of 179ms and a p99 around 203ms (the restore-and-resume core is roughly 49ms), versus a genuine cold boot of about 3 seconds that happens exactly once per plugin version — when you first bake its snapshot — and is then amortized across every install and every invocation. Memory is copy-on-write (MAP_PRIVATE) and the rootfs is a reflink clone, so the Nth instance of a popular plugin doesn't copy gigabytes; it shares the baked pages until it writes. If you'd rather branch one warm plugin VM to fan out N invocations, a same-host fork is 400–750ms (cross-host 1.2–3.5s).

The security payoff of versioning goes beyond speed. Each plugin version is a distinct, immutable snapshot. When the author ships v1.1, you bake a new snapshot from the new code and route new invocations to it — the old v1.0 snapshot is still sitting there, byte-for-byte, if you need to roll back or forensically diff what changed. And because a snapshot is exactly the code you baked, there's no drift: the VM that runs is provably the version you approved, not whatever `npm install` decided to pull at runtime. If v1.1 turns out to be malicious, you flip routing back to the v1.0 snapshot and every new invocation is instantly clean, with a hardware boundary that already contained the bad version to its own disposable guests.

Per-plugin microVM vs. shared runtime vs. WASM

There are three common shapes for where a plugin runs. It's worth laying them side by side rather than picking one dogmatically, because the right answer depends on what your plugins actually need to do:

• Per-plugin microVM — isolation: strongest; each plugin gets its own guest kernel behind a hardware boundary, so a malicious update is trapped in a disposable VM. Capability control: per-plugin secrets, per-plugin egress allowlist, hypervisor-enforced CPU/RAM. Syscalls & network: full — the plugin gets a real Linux kernel, so real filesystem, real subprocess, real sockets, all fenced. Cost: a snapshot restore per invocation (~179ms), near-zero idle. Best for: any plugin that runs real code and may touch the network or user data — the general marketplace case.
• Shared runtime + static review — isolation: weakest; a shared host kernel (or a shared language VM) is the only wall, and it wasn't built to hold adversarial code. Capability control: whatever your process-level sandbox and code review catch, which lags updates. Syscalls & network: broad but hard to fence precisely. Cost: cheapest per call, most expensive per incident. Best for: only trusted first-party plugins, never arbitrary third-party code.
• WASM plugin sandbox — isolation: strong for pure compute; a WASM module has no ambient authority and only the host functions you explicitly grant. Capability control: excellent and fine-grained for the capability model, by design. Syscalls & network: this is the catch — WASM has no native syscalls or sockets; every effect must be brokered through host imports (WASI is still maturing and uneven). Cost: extremely low, microsecond-scale starts. Best for: plugins that are pure functions — parse, transform, score, validate — where 'no real syscalls' is a feature, not a blocker.

The honest read: WASM is genuinely excellent when a plugin is pure compute and you can express every side effect as a host import you control — it starts in microseconds and the capability model is a joy. It gets awkward fast when a plugin needs to shell out, spawn a real process, use a library that assumes a filesystem, or open arbitrary network connections, because none of that exists in the WASM world without you building and vetting a host-function bridge for each capability. Marketplaces where plugins integrate with the real world — call third-party APIs, run existing npm/PyPI packages, execute a subprocess — tend to want the real kernel a microVM provides, fenced by egress rules, rather than a compute sandbox they have to teach how to do I/O. Many mature platforms end up with both: WASM for the pure-function fast path, microVMs for anything that needs a real machine. Verify any specific competitor's isolation claims against their own docs before you trust them.

When you don't need a microVM per plugin

This is real infrastructure, not a free lunch — you own snapshot hygiene, per-plugin capability policy, egress rules, and the plumbing to get inputs in and outputs out. Be honest about whether your threat model calls for it.

• Your 'plugins' are declarative config, not code. If a plugin is a field mapping, a filter expression, or a webhook URL with no arbitrary loops or code, evaluate it in-process — there's nothing Turing-complete to contain.
• Plugins are all first-party. If only your own team ships plugins, a hardened container is likely enough; the malicious-update argument evaporates when you control the updater.
• Plugins are pure functions and WASM fits. If every plugin is compute with no real I/O, a WASM sandbox gives you fine-grained capability control at microsecond cost, and you skip operating a VM fleet.
• A managed FaaS already fits. If your plugin invocations map cleanly onto Lambda or Cloud Functions and you don't need byte-level control of the runtime or egress plane, use one — they run on microVMs under the hood anyway.

Reach for per-plugin microVMs when you run genuinely arbitrary third-party code, when plugins touch user data or the network (so exfiltration and SSRF are in scope), when a malicious post-approval update is a realistic threat (it is), or when compliance demands a hardware boundary between plugins and your data. In those cases snapshot-restore gives you what containers can't — a real kernel boundary per invocation — without the boot tax that used to make per-invocation VMs look absurd. A marketplace plugin is a stranger's code that changes after you stop looking. Contain the version that's running, not the one you approved.