Giving a MicroVM Access to a Customer's Private Network

The request arrives in almost the same words every time. The customer wants your agent, or your hosted app, or your code sandbox, to reach something that lives inside their network — a Postgres instance in a private subnet, an internal HTTP API on RFC1918 space, a legacy thing in a rack in a building. It has no public address, and the whole point of it having no public address is that it should not get one because you showed up.

I'm Ajay; I build PandaStack, a Firecracker microVM platform where every sandbox is its own VM in its own network namespace. This is the design problem I would work through with a customer asking for that, in the order I would work through it: why the obvious answer is bad for both parties, what the right unit of attachment is, how you handle keys for a peer that might live ninety seconds, the snapshot trap that will bite you and is genuinely non-obvious, and the MTU bug that presents itself as "large responses hang" and eats two days.

Why "just allowlist our egress IPs" is bad for both of you

This is where every one of these conversations starts, because it is the cheapest thing to say. You publish a small set of static egress IPs, the customer opens their firewall to those IPs on one port, and everyone goes back to work. It does work, in the narrow sense that packets arrive. It is bad for the customer and it is bad for you, in different ways, and it is worth being able to explain both because the customer is usually only worried about their half.

Their half: the thing they have authenticated is a source IP address, which is to say they have authenticated nothing. Anything that can emit packets from that address is now inside. On a multi-tenant sandbox platform, that address is shared by every tenant on the host, so the customer's firewall rule cannot distinguish their own workload from a stranger's. If your platform runs untrusted, model-generated code — and if you are reading this, it does — then "an allowlisted route into a customer's private subnet, reachable from any tenant's VM" is a sentence that should make you put down your coffee. The customer thinks they granted access to your product. They granted it to your NAT gateway.

Your half is subtler and shows up later. The moment a customer's firewall rule names your egress IPs, those IPs are load-bearing production infrastructure that you cannot change. Not when you move regions, not when you add a host, not when your cloud provider reassigns something, not when you want to run part of the fleet somewhere cheaper. You have quietly signed a contract, with no expiry, whose terms are "these dotted quads are permanent." Multiply by the number of customers who did the change-control paperwork to add them, and you have an infrastructure decision that now requires a coordinated migration with people who do not work for you.

An IP allowlist authenticates a path, not a party. It is a door that opens for whoever is standing in the right hallway — and on a multi-tenant platform, the hallway is public.

There is a third problem that only surfaces during an incident: the allowlist gives neither side an audit trail. When the customer asks which of their internal hosts your platform touched last Tuesday, the honest answer from a shared NAT is "one of our sandboxes did, and I cannot tell you which." That is not a fun sentence to say to a security team, and it is the sentence you will have to say, because you built the design that makes it true.

The unit of attachment is the sandbox's network namespace

Everything good about the alternative depends on having somewhere to attach a tunnel that is not "the platform." On PandaStack each sandbox gets its own Linux network namespace, its own veth pair, and its own tap device carrying the guest. The namespaces are not created on demand at boot — that costs about a tenth of a second of `ip netns add` and `iptables` work you do not have in a create budget — they are pre-allocated as a pool of 16,384 /30 subnets carved out of a private /16, one slot per sandbox, waiting to be claimed. A create claims a warm slot, patches the tap's MAC to match the identity baked into the template snapshot, and restores.

The reason that matters here has nothing to do with boot time. It means there is a real, kernel-enforced network container per sandbox, with its own routing table, its own conntrack view, and its own iptables ruleset, all torn down atomically when the sandbox dies. That is the natural place to attach a VPN, and it is the difference between "our platform is connected to your VPC" and "this one VM is connected to a slice of your VPC." The first is an architecture the customer's security team should reject. The second is one they can reason about.

The isolation between slots has to be enforced explicitly, and it is worth saying out loud because it is the property the whole design leans on. Guests are pre-allocated adjacent /30s in one pool CIDR, the host has IP forwarding on so they can reach the internet, and connected subnets are, by default, routable to each other. So the first rule in the host's FORWARD chain is a DROP for pool-to-pool traffic — inserted first, ahead of the egress ACCEPTs, because iptables is ordered and an appended rule that runs second is a rule that does nothing. Without it, tenant A could scan the pool for tenant B's SSH and app ports. With it, tenant A cannot see tenant B at all, which is exactly the property you need before you let tenant A terminate a tunnel into their employer's network.

Three shapes of private access, compared honestly

There are three designs people actually ship for this, and the choice is less about cryptography than about who holds what credential and for how long. Treat any comparison of specific vendor products as a snapshot in time — verify the current behaviour against their docs before you commit.

• What the customer must open — Egress-IP allowlist: an inbound hole in their perimeter, permanently, for addresses they do not control. WireGuard per-sandbox: one UDP port to a hub they run, or nothing at all if the hub dials out. Mesh agent in guest: outbound to the mesh coordination service, plus trust in a third party's control plane.
• Credential granularity — Egress-IP allowlist: one shared credential (an IP) for every tenant on the platform, with no expiry. WireGuard per-sandbox: one keypair per sandbox, scoped on the hub to a single /32 and a single destination prefix. Mesh agent in guest: one node identity per sandbox, typically issued by an auth key that is itself a longer-lived secret you have to protect.
• Blast radius of a leak — Egress-IP allowlist: total and permanent; whoever can send from that address is inside, and you cannot revoke an IP. WireGuard per-sandbox: one sandbox, until its TTL expires, on one prefix. Mesh agent in guest: one node until revoked, but a leaked pre-auth key mints new nodes at will, so the key is the real asset.
• Time to attach a new sandbox — Egress-IP allowlist: zero at runtime, weeks the first time (change control). WireGuard per-sandbox: milliseconds, if your control plane can mint a key and register a peer; it is two API calls and a file write. Mesh agent in guest: seconds, because the node has to register and converge with the coordination service before traffic flows.
• Snapshot safety — Egress-IP allowlist: unaffected, since there is no secret in the guest. WireGuard per-sandbox: dangerous unless you are disciplined; a baked key is a cloned identity across every restore. Mesh agent in guest: the same hazard, larger, because agents cache node keys and state on disk and will happily resume as a duplicate of themselves.
• Audit: which sandbox touched which host — Egress-IP allowlist: unanswerable from the customer's side; everything is one NAT address. WireGuard per-sandbox: answerable exactly, because the tunnel IP is the sandbox identity and the hub logs per-peer counters. Mesh agent in guest: answerable, usually with good tooling, on the vendor's terms and in the vendor's retention window.
• Who runs the control plane — Egress-IP allowlist: nobody, which is why it is popular. WireGuard per-sandbox: you do, and it is real work: key minting, peer registration, expiry, reconciliation after a host disappears. Mesh agent in guest: a third party, which is a genuine operational win and a genuine dependency, and both halves belong in the security review.
• Where it breaks first — Egress-IP allowlist: at the security review, or during the incident nobody can reconstruct. WireGuard per-sandbox: at MTU, and at peers that outlive their sandbox. Mesh agent in guest: at guest footprint and startup latency, since you are now booting a daemon that must reach a coordinator before the workload can do anything.

My honest read: WireGuard per sandbox is the right default for a platform that already has per-sandbox network namespaces, because you own the primitives and the credential lifetime can be made to match the sandbox lifetime exactly. A mesh product is the right default for a team that does not want to build a hub, an expiry reconciler, and a peer registry — those are the three pieces you are actually signing up for, and none of them are hard, but all of them are yours forever. The allowlist is the right default for nobody, and it is what most platforms ship, because it is the only one of the three that requires no engineering.

What the config actually looks like

WireGuard is small enough that the whole design fits in one file, which is most of why it belongs in a microVM. It is in the guest kernel, it has no daemon, no state machine to babysit, and no certificate infrastructure — a peer is a public key and a list of prefixes it is allowed to use. That last part, cryptokey routing, is the feature doing the security work here, and it is worth reading the config with that in mind.

# /etc/wireguard/wg0.conf -- written INTO the guest after restore, never
# baked into the template. One file, one sandbox, one identity, and a
# lifetime measured in minutes.

[Interface]
# Minted per sandbox by the control plane at create time. Mode 0600. It
# exists for exactly as long as this VM does.
PrivateKey = <freshly generated, per sandbox>

# A /32 out of a range you reserve for ephemeral peers. Deliberately NOT
# part of the customer's subnet: give tunnel endpoints their own space so
# an audit log can tell a sandbox from an employee laptop at a glance.
Address = 100.96.4.17/32

# The number that stops "large responses hang". See the MTU section --
# this is the single most common way this setup fails in a microVM.
MTU = 1420

# Note what is NOT here: no DNS =. Setting it sends every lookup the guest
# makes over the tunnel, which is full-tunnel resolution wearing a
# split-tunnel costume, and it leaks your workload's browsing to the
# customer's resolver. Resolve internal names explicitly instead.

[Peer]
# The customer's hub: their VPC gateway instance, their on-prem
# concentrator, whatever they already run.
PublicKey = <hub public key>

# The hub authenticates us by our key; the PSK is a second, symmetric
# factor that also happens to be the post-quantum hedge. It is one extra
# 32-byte value per peer. Use it.
PresharedKey = <per sandbox>

Endpoint = vpn.customer.example:51820

# SPLIT TUNNEL, and this line is the entire security argument. AllowedIPs
# on the client is both a routing table and an inbound filter: packets
# arriving from the hub with a source outside these prefixes are dropped
# by the kernel, not by a policy you remembered to write.
AllowedIPs = 10.42.7.16/28, 10.42.9.5/32

# We sit behind two layers of NAT (a per-netns SNAT, then the host
# masquerade), so the hub can never dial us. We hold the mapping open.
PersistentKeepalive = 25

Bringing it up inside the guest is four commands, and I would write them out rather than reach for `wg-quick`. Not because `wg-quick` is bad, but because it also rewrites `resolv.conf` and can install a default route, and in a sandbox both of those are decisions you want to be making deliberately rather than inheriting.

#!/bin/sh
# In-guest tunnel bring-up. Runs AFTER the sandbox is restored and the
# config has been injected -- never at bake time.
set -eu

CONF=/etc/wireguard/wg0.conf   # 0600, written by the control plane
TUN_IP=100.96.4.17

ip link add dev wg0 type wireguard

# setconf reads [Interface] PrivateKey and the [Peer] blocks. It does NOT
# read Address or MTU -- those keys are wg-quick's, not wg(8)'s. This is
# precisely why hand-rolled setups end up at MTU 1500 and then spend two
# days blaming the customer's firewall.
wg setconf wg0 "$CONF"

ip address add "$TUN_IP/32" dev wg0
ip link set mtu 1420 up dev wg0

# SPLIT TUNNEL: route only the granted prefixes. No default route, so the
# guest's ordinary internet traffic never enters the customer's network.
ip route add 10.42.7.16/28 dev wg0
ip route add 10.42.9.5/32   dev wg0

# Proof of life, and the check your orchestration should gate on. A
# nonzero timestamp means the hub accepted our key AND our clock. A zero
# here with tx bytes climbing and rx bytes flat is the frozen-clock
# failure described below -- we are transmitting into silence.
wg show wg0 latest-handshakes

# The MTU test that actually tests something. Ping is 64 bytes and will
# happily succeed across a broken path MTU; pull a response big enough to
# need full-size segments.
curl -sS --max-time 10 -o /dev/null -w 'bytes=%{size_download}\n' \
  http://10.42.7.20:8080/large-response

Key lifecycle for a peer that lives ninety seconds

Here is where sandbox platforms diverge from every VPN tutorial ever written, all of which assume a peer is a laptop that exists for three years. On a snapshot-restore platform a create costs a couple of hundred milliseconds, so a customer's workload might be a thousand sandboxes over an afternoon, each alive for under a minute. Static keys are not merely inelegant in that world; they are wrong, because a static key is a credential whose lifetime you have deliberately decoupled from the thing it authorises.

The rule I would hold to: the tunnel credential's lifetime is the sandbox's lifetime. Both, or neither. Concretely that means four things.

1. Mint the keypair per sandbox, at create time, on the control plane. Generating in the guest is defensible, but then you need a channel to get the public half back before the handshake can work, and you are trusting the guest's entropy in the seconds immediately after a snapshot restore — which is exactly the moment it is least trustworthy. Minting outside is simpler, auditable, and one less thing to reason about.
2. Register the peer on the hub before the guest tries to connect, with AllowedIPs on the hub side scoped to that one /32. The hub's AllowedIPs is its cryptokey routing table: it is what stops a sandbox with a valid key from claiming another sandbox's tunnel address. Skip it and you have per-sandbox keys with platform-wide authority, which is the allowlist problem again with more steps.
3. Give the peer registration its own expiry, independent of teardown, and set it to the sandbox TTL plus a small margin. Teardown-driven revocation is the happy path, and the happy path does not run when a host disappears. An expiry that fires on its own is what makes the failure mode "a peer that cannot hand shake" instead of "a standing credential into a customer's VPC that nobody remembers creating."
4. Revoke on teardown anyway, and treat that revocation as a security control rather than as cleanup — which in practice means it needs a retry, a reconciler that sweeps peers with no live sandbox, and an alert when the two sets disagree. If your platform can leak a network namespace, it can leak a peer, and the second one is worse.

The reconciler is the piece people skip, and it is the piece that matters. You already know how to build it if you run a fleet: it is the same shape as the sweeper that marks sandboxes failed when their host stops heartbeating. Enumerate the hub's peers, enumerate live sandboxes, remove the difference. Run it on a timer and on every teardown. The timer is what saves you the day a host vanishes without unwinding anything it owned.

The snapshot trap: a baked key is a cloned identity

This is the part I would put in bold in a design review, because it is genuinely non-obvious and the symptoms do not point at the cause. On a platform where every create is a snapshot restore, a template is a frozen VM — memory, devices, disk — and every sandbox from that template starts life as a byte-identical copy of it. Whatever secret was in that VM when you froze it is now in every clone, simultaneously, forever.

So if you bake a template with WireGuard "already configured" because it makes the create path simpler, you have shipped one private key to every sandbox on the platform. On the hub, all of them are the same peer. WireGuard is a single-session protocol per peer: two clones handshaking with the same static key do not get two tunnels, they get one tunnel that thrashes as each new handshake replaces the last. The user-visible symptom is not "security incident." It is intermittent connectivity that gets worse under load, which is the single worst class of bug to be handed, because it looks like a network problem and it is an identity problem.

There is a second, sneakier version of this that bites even when you do everything right, and it comes from the clock. WireGuard's handshake carries a TAI64N timestamp, and the responder rejects any handshake whose timestamp is not greater than the greatest one it has already seen for that peer. That is deliberate replay protection and it is excellent — right up until you restore a Firecracker snapshot, because a restored guest resumes with CLOCK_REALTIME frozen at the instant the snapshot was taken. Firecracker microVMs have no RTC on x86 and templates do not run NTP, so nothing in the guest corrects it on its own.

That is not a theoretical concern for us; the same frozen clock shipped as a live incident here in a different disguise. A seed baked in June, an upstream host rotating onto a certificate issued in July, and every app deploy's `git clone` failing with a certificate-verification error, because the guest believed it was still June and the certificate was not yet valid. The fix — re-syncing the guest wall clock over the exec bridge on every restore, wake, and resume path — is the same fix WireGuard needs, for the same reason. Any guest that is going to speak a protocol with timestamp-based replay protection needs its clock corrected before it speaks, not after somebody files a bug about handshakes silently failing.

The general rule this is a special case of: a snapshot is a photograph of everything, and anything in the guest whose correctness depends on being unique or on moving forward — private keys, session counters, seeded PRNG state, machine IDs, the clock — is now duplicated and stopped. Inject uniqueness after restore, never before. That means the tunnel config is written into a running guest and the interface is brought up as part of provisioning, which costs you a file write and a handful of commands on a create path that is already measured in hundreds of milliseconds. It is a very cheap insurance premium.

MTU, or: why big responses hang and small ones don't

Every WireGuard deployment meets this bug and microVMs meet it harder, so it is worth understanding rather than cargo-culting the number. WireGuard encapsulates each packet in UDP and adds its own header and authentication tag — 32 bytes of WireGuard, 8 of UDP, and 20 or 40 of IP header depending on whether the outer path is v4 or v6. If the guest's virtio-net interface is the usual 1500 and you leave `wg0` at 1500 too, a full-size packet plus overhead exceeds the path MTU the moment it leaves. `wg-quick` defaults to 1420, which is 1500 minus 80, and the 80 is the worst case: 40 bytes of IPv6 header plus 8 of UDP plus 32 of WireGuard. On a pure IPv4 path the arithmetic gives you 1440. Use 1420 anyway, because you rarely know the whole path, and 20 bytes of headroom is not worth the outage.

The reason this is such a miserable bug is the shape of the symptom. Small packets fit and work perfectly. Your handshake succeeds. `ping` succeeds — it is 64 bytes. `curl` on a small JSON endpoint succeeds. TCP's connection setup succeeds, because SYN packets are tiny. Then the first response large enough to need full-size segments simply stops, and the connection sits there until something times out. Everything you would naturally reach for to test the tunnel reports that the tunnel is fine, because everything you would naturally reach for is small.

What should rescue you is Path MTU Discovery: the router that cannot forward the oversized packet sends back an ICMP "fragmentation needed" message, the sender shrinks its segments, and traffic resumes. What actually happens is that somebody's firewall drops all ICMP because a checklist from 2003 said to, and the sender never hears about it. It just keeps retransmitting a packet that will never arrive. This is the classic PMTUD black hole, and inside a microVM you have added hops — guest, tap, veth, a host masquerade — that give it more places to hide.

One microVM-specific wrinkle worth knowing about. Guest packets on our setup get source-rewritten twice on the way out: once inside the sandbox's own namespace, from the baked guest address to the slot's unique veth address, and once at the host's masquerade. The first rewrite exists because the guest IP is baked into the template snapshot and is therefore identical across every sandbox from that template — without rewriting it to something unique per slot, conntrack entries would collide between tenants. The practical consequence for WireGuard is that your UDP flow is behind two NATs and the hub can never initiate. `PersistentKeepalive` is not optional decoration in that setup; it is what keeps both NAT mappings alive so the hub's replies have somewhere to go.

Split tunnel, plus default-deny, is the only sane default

Full tunnel means the sandbox's default route goes into the customer's network: everything it does, including `pip install` and every API call to your own control plane, transits their infrastructure and egresses from their perimeter. Split tunnel means only the granted prefixes go over `wg0` and everything else takes the ordinary path. People reach for full tunnel because it feels more secure — one path, one policy, one place to inspect.

It is not more secure; it moves your problem into someone else's network and makes it their problem. Untrusted, model-generated code now egresses from the customer's IP space, which means their abuse desk fields the complaints, their DLP sees your traffic, their proxy logs fill with your package downloads, and the incident where a sandbox does something regrettable is an incident with their name on the outbound packets. It also makes your platform's reliability depend on their VPN concentrator, which is a dependency neither of you priced in.

Split tunnel with a narrow AllowedIPs is the right shape, and the reason it is genuinely safer is that WireGuard's AllowedIPs is not just a route — it is an inbound filter too. A packet arriving on the tunnel with a source address outside the configured prefixes is dropped by the kernel before anything in userspace sees it. You get "this sandbox may reach these three hosts, and may only be reached by these three hosts" from one line of config, enforced in the kernel, with no policy engine to misconfigure.

The other half is the general egress policy, and I want to be precise about what we do rather than what I would recommend, because those are not the same list. Our guest egress today is default-allow with a denylist: cross-tenant traffic dropped, the entire link-local metadata range dropped, the well-known Stratum mining ports dropped — that last one because free-tier miners are a recurring abuse vector and tripping a cloud provider's crypto-mining detector can get the whole project flagged. That is the right default for a general-purpose sandbox platform where users legitimately need to reach arbitrary package registries and APIs. It is not the right default for a sandbox that is attached to a customer's private network. For that case, the shape you want is inverted: a default-deny egress policy on the namespace, an explicit allowlist for what the workload genuinely needs, and the tunnel prefixes on top. Denylists are for abuse prevention at scale; allowlists are for the workload you promised a customer you would constrain.

A denylist is a statement about what you have thought of. An allowlist is a statement about what you have agreed to. Only one of those is worth writing into a contract.

Auditing which sandbox touched which internal host

The customer will ask this question, and if you are lucky they ask during procurement rather than during an incident. The instinct is to answer it from the network — capture traffic, correlate flows — and that instinct is wrong twice over. It is wrong because packet capture at scale is expensive and privacy-hostile, and it is wrong because on this architecture it does not work: when the tunnel terminates inside the guest, the host sees an encrypted UDP flow to the hub's endpoint and nothing else. The inner destinations are, correctly, invisible to you.

So answer it from the two ends you do own. The first is provisioning: when you mint a peer, you know the sandbox ID, the tenant, the public key, the tunnel /32, and the exact prefixes you granted. Emit that as a structured event, not a log line, and you have a durable record of what was authorised and when — which is most of what an auditor is asking for. The second is the hub, where `wg show wg0 dump` gives you one line per peer with the endpoint, allowed IPs, last handshake, and cumulative byte counters. Sample it on an interval and you have per-sandbox usage into the customer's network without inspecting a single packet.

# HUB SIDE. wg(8) already keeps the accounting you need: one tab-separated
# line per peer -- pubkey, psk, endpoint, allowed-ips, latest handshake,
# rx bytes, tx bytes, keepalive. Scrape on an interval, join to the
# provisioning event by pubkey, and per-sandbox attribution falls out.
wg show wg0 dump | awk -F'\t' 'NR>1 {
  printf "peer=%s endpoint=%s allowed=%s handshake=%s rx=%s tx=%s\n",
         $1, $3, $4, $5, $6, $7
}'

# Expire a peer. Run this on teardown AND on a timer -- the timer is what
# covers the host that disappeared without unwinding what it owned.
wg set wg0 peer "$PUBKEY" remove

# Reconcile: peers the hub still knows about with no live sandbox behind
# them. This set should always be empty. When it isn't, you have standing
# credentials into a customer's network and you want to know today.
comm -23 \
  <(wg show wg0 peers | sort) \
  <(curl -sS -H "Authorization: Bearer $PANDASTACK_API_KEY" \
      https://api.pandastack.ai/v1/sandboxes \
    | jq -r '.sandboxes[].metadata.wg_pubkey | select(. != null)' | sort)

# HOST SIDE, for completeness and to set expectations: the per-sandbox /30
# means conntrack is already scoped per tenant, so you can attribute the
# OUTER flow precisely. You cannot see inside it -- which is the correct
# amount of visibility for a tunnel into someone else's network.
conntrack -L -s 10.200.0.6 2>/dev/null | grep 'dport=51820'

If you genuinely need per-destination visibility on your side — some compliance regimes ask for it — the answer is to move the tunnel out of the guest and terminate it in the sandbox's network namespace instead. Then the guest routes plaintext to the namespace, the namespace encrypts, and the host can log inner destinations per sandbox with ordinary connection tracking. That is a real design with a real trade: you gain visibility and lose the property that the guest holds its own key, which means a control-plane compromise now reaches the tunnel directly. I would default to terminating in the guest and only move it if a customer contract requires the inner logs.

Putting it together

The whole flow from the caller's side is: mint, register, create, inject, verify, work, revoke. The verify step is the one people leave out and the one that turns a silent failure into a clean error — if the handshake did not happen, the job must not run, because otherwise you get a connection timeout thirty seconds into someone's workload and no indication that the tunnel was the problem.

import subprocess
from pandastack import Sandbox

# Mint on the control plane, not in the guest: no channel needed to get the
# public half back, and no dependence on guest entropy in the seconds right
# after a snapshot restore -- which is exactly when it is least trustworthy.
def mint_keypair() -> tuple[str, str]:
    priv = subprocess.run(["wg", "genkey"], capture_output=True,
                          text=True, check=True).stdout.strip()
    pub = subprocess.run(["wg", "pubkey"], input=priv, capture_output=True,
                         text=True, check=True).stdout.strip()
    return priv, pub

TTL = 900  # the tunnel's lifetime IS the sandbox's lifetime. Both, or neither.

priv, pub = mint_keypair()
tunnel_ip = allocate_tunnel_ip()      # a /32 from your ephemeral peer range

sbx = Sandbox.create(
    template="agent",
    ttl_seconds=TTL,
    # The audit trail starts HERE, not in a packet capture. Whatever this
    # sandbox reaches inside the customer's network, you can name the peer.
    metadata={"tenant": "acme", "wg_pubkey": pub, "wg_ip": tunnel_ip},
)

try:
    # Register on the hub BEFORE the guest hands shake, with AllowedIPs
    # scoped to this one /32. That is the hub's cryptokey routing table --
    # it is what stops a valid key from claiming another sandbox's address.
    hub.add_peer(public_key=pub, allowed_ips=f"{tunnel_ip}/32", expires_in=TTL + 60)

    # Inject post-restore. Nothing secret was ever in the baked snapshot.
    sbx.filesystem.write(
        "/etc/wireguard/wg0.conf",
        WG_CONF_TEMPLATE.format(private_key=priv, address=tunnel_ip),
    )
    sbx.exec("chmod 600 /etc/wireguard/wg0.conf", check=True)
    sbx.exec("sh /usr/local/bin/wg-up.sh", timeout_seconds=30, check=True)

    # FAIL CLOSED. `latest-handshakes` prints "<pubkey><TAB><unix-ts>", and a
    # zero means the hub never accepted us -- wrong key, wrong PSK, or the
    # frozen-clock replay rejection. Do not start the job on a dead tunnel.
    hs = sbx.exec("wg show wg0 latest-handshakes", timeout_seconds=10).stdout
    if not hs.strip() or hs.split()[-1] == "0":
        raise RuntimeError("no WireGuard handshake -- refusing to run")

    out = sbx.exec(
        "psql -h 10.42.9.5 -U readonly -c 'select count(*) from orders'",
        timeout_seconds=60,
    )
    print(out.stdout)

finally:
    # Deregistration is not cleanup, it is the security control. A peer that
    # outlives its sandbox is a standing credential into a customer's VPC.
    # Needs a retry and a reconciler behind it -- see the audit section.
    hub.remove_peer(pub)
    sbx.kill()

One thing that is not in that snippet and should be in yours: the customer-facing story about what happens when the tunnel is down. A sandbox that cannot reach the customer's database should fail with a clear, attributable error rather than a generic connection timeout, because otherwise every VPN problem arrives as a support ticket about your product being slow. Gate the workload on the handshake, surface the tunnel state in whatever status your API exposes, and you convert an entire category of confusing incident into a legible one.

What I'd tell a customer asking for this

That we are not going to ask them to open their perimeter to a set of shared addresses, because that grants access to a hallway rather than to a party, and on a multi-tenant platform the hallway is public. That instead each sandbox that needs their network gets its own key, its own tunnel address, its own narrow list of prefixes, and an expiry that fires whether or not our teardown code runs. That the key exists only inside a running VM, never in an image, because an image gets cloned and a cloned key is both an outage and a cryptographic failure. That they will be able to see, per sandbox, what handshook and how much moved. And that when the sandbox goes away — which for most of them is a matter of minutes — the credential goes away with it.

None of that is exotic. It is one kernel module that has been in Linux for years, a hub they probably already run, a key minter, and a reconciler. The engineering is small. The discipline is the whole thing: never bake a secret, never let a credential outlive its subject, never grant a prefix you cannot name, and never trust a tunnel you have not verified before you start the work.