Safely running pip install from LLM-generated code

Somewhere in your agent loop, a model decides it needs a library it doesn't have. It emits `pip install some-package`, your executor runs it, and a few seconds later the code does `import some_package` and carries on. Feels routine. But you just executed arbitrary third-party code — code the model chose, fetched from PyPI, on demand — and the dangerous part happened before your `import` ever ran. This post is about that gap: why `pip install` from LLM-generated code is code execution, not a download, and the microVM pattern that makes it safe to let an agent install whatever it wants.

I'm Ajay — I built PandaStack, a Firecracker microVM platform for exactly this kind of untrusted execution. I'll be concrete about the threat and honest about where pinning and hashes help versus where you actually need an isolation boundary.

pip install is curl | sudo bash wearing a trench coat

The mental model that gets people hurt is "install downloads the package, import runs it." That's wrong on the first half. A source distribution can run arbitrary Python at install time. When pip builds an sdist it executes the package's build backend — historically `setup.py`, now often a PEP 517 hook in `pyproject.toml` — and that code runs as your user, with your environment, your network, and your credentials, the moment you type install. You never reach `import`. There is nothing to review first, because the malicious behavior fires during the build.

• setup.py — plain Python that runs on install. A `setup.py` can shell out, read ~/.aws/credentials, and POST it somewhere before pip prints "Successfully installed".
• PEP 517 build backends — pyproject.toml can point the build to arbitrary code (a custom backend, a build hook). Same execution, newer wrapper.
• Post-install / import-time payloads — even a wheel (no build step) runs whatever is at the top of its modules the instant you import it. The model asked to import it, so that's coming too.
• Dependency chains — you audited the package; did you audit its 40 transitive dependencies, each of which also gets to run install-time code?

Why an LLM in the loop makes this sharply worse

A human developer typing `pip install requests` has, at minimum, typed a name they meant. An LLM picks the package name from a distribution over tokens. It will confidently install `python-requests`, `requestss`, or `reqwests` — names that don't exist as the library it wants, but that a typosquatter has absolutely registered on PyPI hoping for exactly this. Registering `numpyy` and waiting for a fat-fingered install (human or model) is a real, ongoing attack class.

It gets worse under prompt injection. If your agent reads a web page, a GitHub issue, or a tool result that an attacker controls, that text can steer the model toward "you'll need to `pip install helpful-utils` for this." The model obliges. Now an attacker chose the package, the model ran the install, and the install ran their code. You've turned a content channel into a code-execution channel, and the only thing standing between that and your host is whatever isolation you put around the install step.

The safe assumption is that any package name an LLM emits might be attacker-influenced and might run hostile code at install time. Design so that assumption costs you a disposable VM, not your infrastructure.

Why a shared-kernel container is a weak boundary here

The instinct is "run the install in a container." Better than your host process, and fine for code you trust — but for genuinely untrusted install-time code it's a soft boundary. A container shares the host kernel. The install code you're worried about runs as a normal process against that shared kernel, and a kernel bug or a container-escape primitive puts it on the host and, in a multi-tenant setup, into other tenants. That's not theoretical enough to ignore: AWS, Google, and others moved untrusted workloads off plain containers and onto microVMs or gVisor precisely because a shared kernel is a large, attackable surface.

There's a second, more mundane failure. Containers are usually built with ambient access to things the install code should never see — the host's Docker socket bind-mounted in, cloud metadata at 169.254.169.254 reachable, an env var full of API keys inherited from the parent process. A malicious `setup.py` doesn't need a kernel exploit to ruin your day if it can just read a credential that was sitting right there. The most common real-world leak is boring: ambient network plus an inherited secret, not an exotic escape.

The options, ranked for untrusted installs

Where you run an untrusted `pip install`, from least to most contained:

• Host pip (into your app's environment) — no boundary at all. Install-time code runs as you, sees your secrets, mutates your dependency tree. Never do this with a model-chosen package.
• A venv on the host — isolates which packages are importable, not what install-time code can do. A venv is a PYTHONPATH trick, not a security boundary; the setup.py still runs as your user against your host.
• A container — real process isolation and good hygiene, but a shared kernel and often ambient network/credentials. Fine for trusted builds; a soft boundary for untrusted install-time code.
• A disposable microVM — the install runs behind a hardware virtualization boundary with its own guest kernel, no host credentials present, and egress you control. If the package is hostile, the blast radius is one VM you were going to delete anyway.

The pattern: install inside a disposable microVM

The durable shape has four properties: a hardware isolation boundary (a microVM with its own kernel, not a shared one), an ephemeral environment (fresh per session or per install, destroyed after, so nothing persists forward), no host credentials in the guest, and controlled egress so a package can't quietly exfiltrate. On PandaStack you get the first two by construction and enforce the rest per sandbox. Here's the loop with the Python SDK — create a throwaway VM, install the model's package inside it, run the user code, read the result back, destroy it:

from pandastack import PandaStack

ps = PandaStack()  # reads PANDASTACK_API_KEY from the environment

# The model decided it needs this package and wrote code that imports it.
pkg = "some-pkg"            # attacker-influenceable: treat as hostile
user_code = """
import some_pkg
print(some_pkg.compute(2, 3))
"""

# One disposable microVM for this install. ttl reaps it if we crash.
sb = ps.sandboxes.create(template="code-interpreter", ttl_seconds=600)
try:
    # pip install runs THIRD-PARTY CODE (setup.py / build hooks) right here,
    # inside the guest kernel, with no host credentials and controlled egress.
    r = sb.exec(f"pip install --no-cache-dir {pkg}", timeout_seconds=120)
    if r.exit_code != 0:
        raise RuntimeError(f"install failed / blocked:\n{r.stderr}")

    # Now the import (also third-party code) runs in the same contained VM.
    sb.filesystem.write("/workspace/run.py", user_code)
    out = sb.exec("python3 /workspace/run.py", timeout_seconds=60)
    print("exit:", out.exit_code)
    print(out.stdout)

    # Pull any artifact back out through the API, not a host mount.
    # data = sb.filesystem.read("/workspace/result.json")
finally:
    sb.delete()  # VM and everything the package did to it are gone

If `some-pkg` was a typosquat whose `setup.py` tried to read credentials and phone home, here's what it found: no `~/.aws`, no inherited API keys, no metadata endpoint reachable, and egress restricted so the exfil POST never leaves. Whatever it did to the filesystem dies with `sb.delete()`. The worst case is a wasted sandbox — which is the whole point of making sandboxes cheap.

That cheapness is what makes per-install isolation practical rather than a nice idea. On PandaStack every create restores a baked snapshot on demand — p50 179ms, p99 ~203ms, with the snapshot-restore step itself around 49ms (a true first cold boot, before any snapshot exists, is ~3s). So spinning up a fresh VM just to run one sketchy install isn't a boot-time tax you avoid by reusing environments across trust domains; it's fast enough to do every time. If you want each install to start from an identical known-good base, snapshot a configured sandbox once and fork it — a same-host fork is 400–750ms (cross-host 1.2–3.5s) and shares memory copy-on-write. And there's plenty of address space to do this at scale: each agent pre-allocates 16,384 /30 subnets, so per-VM network isolation isn't the bottleneck.

Controlling egress so a package can't phone home

Isolation contains a compromised guest; egress control decides whether it can talk to the outside while contained. The tension is obvious: `pip install` needs to reach PyPI, but a malicious build hook wants to reach an attacker's server. You can't naively block all egress or the install fails. The workable posture is default-deny with a narrow allowlist for the package index, so the install resolves but the arbitrary code in the build step can't open a socket to somewhere you didn't sanction.

• Point pip at an index you control (a private mirror or proxy) and allow egress only to that host — the install works, everything else is denied.
• Deny the cloud metadata endpoint (169.254.169.254) unconditionally; nothing an untrusted package installs has any business there.
• Keep the guest's outbound default-deny; open only what the task genuinely needs, per sandbox, and close it when the sandbox dies.
• Assume DNS is an exfil channel too — an allowlist that only names your mirror shuts down cute tricks like encoding stolen data into lookups.

Better: rarely pip-install untrusted things live at all

The safest untrusted install is the one you don't do at runtime. PandaStack's code-interpreter template bakes the common scientific-Python stack — pandas, numpy, scipy, scikit-learn, matplotlib, and friends — into the snapshot, so it's already importable on a fresh VM with zero install step. Most of the time the model reaches for a library that's already there, no `pip install` fires, and there's no install-time code to worry about because you controlled what went into the image.

For the long tail — a package that genuinely isn't baked — you still fall back to the pattern above: install it live, but inside the disposable, credential-free, egress-controlled VM, so the fallback is contained rather than dangerous. And if your agent keeps reaching for the same handful of extra packages, bake your own template with them included and stop installing them at runtime entirely. The two-tier posture — pre-baked for the common case, isolated live-install for the tail — is what lets you say yes to "the agent can install whatever it needs" without that being a synonym for "the agent can run whatever it's told, as you."

When is none of this necessary? If you fully control the requirements file and it isn't model- or user-influenced, a pinned, hash-checked install into a normal environment is fine — don't spin up a VM to install dependencies you chose and trust. The whole apparatus here is for the case that actually bites: a package name that arrived at runtime, from a model, possibly steered by an attacker. For that case, treat the install as hostile code execution, run it in a VM you're happy to throw away, and let the blast radius be a sandbox instead of your fleet.