A plain-English glossary of sandbox and microVM terms

I have now had the same conversation about thirty times. Someone says "sandbox" and means a Python interpreter with a few builtins removed. Someone else says "sandbox" and means a hardware-isolated virtual machine with its own kernel. Both are using the word correctly, in the sense that both usages are established, and the gap between them is the difference between a speed bump and a wall. Nobody notices until a security review.

I build PandaStack, a Firecracker microVM platform where every sandbox create is a snapshot restore. So I spend most days inside this vocabulary, and I have watched it confuse smart people repeatedly — not because the concepts are hard, but because the words were reused. "Snapshot" means two unrelated artifacts. "Cold start" and "cold boot" sound like synonyms and are not. "Serverless" tells you nothing whatsoever about who your code is running next to.

So this is the glossary I wish existed when I started. Forty-eight terms, grouped, each one written to stand alone if you land on it from a search. Where a term is routinely misunderstood I say so explicitly, because the misunderstanding is usually more useful than the definition.

Isolation and runtimes

Sandbox

An environment where you run code you do not fully trust, constrained so that what it does cannot hurt anything outside it. That is the whole idea, and the word says nothing at all about where the constraint is enforced. It might be a language-level restriction (a Python interpreter with builtins stripped, a JavaScript realm), an OS-level one (seccomp filters, namespaces, a container), or a hardware-level one (a virtual machine with its own kernel, isolated by CPU virtualization extensions).

The distinction that matters: ask what the attacker has to break, and how much code sits at that boundary. A language sandbox is broken by any interpreter bug — and interpreters are enormous. An OS sandbox is broken by any Linux kernel bug reachable through the syscalls you allow, and the kernel is a few tens of millions of lines. A VM sandbox is broken by a hypervisor bug, and a minimal VMM exposes a handful of virtual devices. Those are three wildly different amounts of attack surface being described by one noun.

The way people get it wrong: treating "we run it in a sandbox" as a complete answer in a security review. It is the start of the question, not the end of it. If someone tells you their product sandboxes untrusted code, the only useful follow-up is "at which boundary?"

Container

A process — or a small group of them — running on the host kernel, with its view of the world narrowed by Linux namespaces (its own PID numbering, mount table, network stack, hostname) and its resource usage capped by cgroups. There is no machine in there. There is no second kernel. There is a process, wearing a costume.

What actually matters: containers share the host kernel. Every syscall your container makes is executed by the same kernel that runs every other tenant's container. That is why containers start in milliseconds and cost almost nothing, and it is also why a kernel privilege-escalation bug is a container escape. Both facts come from the same architectural choice.

The way people get it wrong: reasoning about containers as if they were lightweight VMs. They are not small machines, they are constrained processes, and once you internalise that a lot of surprising behaviour stops being surprising — why the container sees the host's kernel version, why it sees host CPU count unless you're careful, why a kernel module loaded anywhere is loaded everywhere.

Virtual machine

A complete synthetic computer: virtual CPUs, a block of memory the guest believes is physical RAM, virtual devices, and its own operating system kernel booted on top of all of it. The guest kernel handles the guest's syscalls. The host kernel never sees them. The boundary is enforced by the CPU itself through hardware virtualization extensions, with the hypervisor mediating the few things that have to cross.

Set against a container: the interface between tenant and host. A container's interface is the Linux syscall surface — hundreds of calls, each with arguments, each a potential bug. A VM's interface is the virtual hardware — a handful of virtio queues and some control registers. Smaller interface, stronger boundary, more overhead. That trade is the entire container-versus-VM argument in one sentence.

MicroVM

A virtual machine with almost all the hardware emulation deleted. No BIOS or UEFI, no PCI bus, no emulated legacy devices, no video, no USB, no floppy controller that a 2026 machine somehow still had. The kernel is loaded directly by the VMM, and the device model is a short list of virtio devices: block, net, vsock, maybe a random number generator.

The useful distinction: this is not a smaller VM, it is a VM with less to initialise and less to attack. A conventional VM spends most of its boot pretending to be a PC from 1998 so that unmodified operating systems will recognise it. Delete that pretence, require a modern virtio-aware guest, and boot drops to tens of milliseconds while the code exposed to a hostile guest drops by orders of magnitude.

Hypervisor and VMM

Two halves of the same job that get called by each other's names constantly. The hypervisor is the privileged part that actually creates virtual CPUs and enforces memory isolation, using the processor's virtualization extensions. The VMM — virtual machine monitor — is the userspace program that sets up the VM, emulates the devices, provides the API, and pumps I/O.

The thing to hold onto on Linux: KVM is the hypervisor, and it is a kernel module. Firecracker, QEMU, and Cloud Hypervisor are VMMs, and they are ordinary userspace processes that talk to KVM through ioctls on /dev/kvm. When someone says "Firecracker is a hypervisor," they usually mean the pair. It matters because the security properties belong to different components: KVM enforces the isolation, the VMM defines how much surface the guest gets to poke at.

KVM

Kernel-based Virtual Machine: the Linux subsystem that turns the kernel into a hypervisor. It exposes /dev/kvm, and a userspace process opens it, asks for a VM, asks for vCPUs, hands over a region of its own address space to serve as guest RAM, and then runs each vCPU in a loop. When the guest does something the hardware cannot handle alone, control returns to userspace with a reason code, and the VMM emulates whatever it was.

The way people get it wrong: assuming KVM needs specific hardware you might not have. Any x86-64 with VT-x or AMD-V, and any ARM64 with EL2 available, will do it. What you actually can't do is run KVM inside a VM whose host declines to expose the virtualization extensions — which is why running Firecracker on a laptop is a nested-virtualization question, not a KVM question.

Firecracker

An open-source VMM written in Rust, originally built at AWS for Lambda and Fargate. It runs microVMs on KVM, exposes a REST API over a Unix domain socket rather than a command line full of flags, and — the part that makes it interesting for this kind of platform — supports full snapshots of a running guest and restoring them into a fresh process.

Compared with QEMU: Firecracker deliberately cannot do most things. No live migration, no PCI passthrough, no GPU, no arbitrary device model, no BIOS. It is a specialised tool for running many short-lived guests densely and safely, and it is a poor general-purpose VMM, on purpose.

Jailer

A small companion binary shipped with Firecracker that sets up a restricted environment and then execs the VMM inside it: a chroot, its own namespaces, a cgroup, dropped capabilities, an unprivileged uid. The VMM then runs with the minimum privilege it needs and nothing else.

The distinction that matters: the jailer protects the host from a compromised VMM, not the host from the guest. The guest is already contained by hardware virtualization. The jailer is the second wall, for the scenario where a guest found a bug in the VMM's device emulation and now controls a host process — it wants that process to be as boring and powerless as possible.

gVisor

A userspace kernel. gVisor intercepts the syscalls a sandboxed process makes and services most of them itself, in Go, in userspace, rather than passing them to the host kernel. You get a much narrower host kernel interface than a plain container without paying for a full guest kernel and virtual hardware.

The part worth holding onto: gVisor reimplements the Linux ABI, which means two things. Compatibility is good but not total — programs that reach for exotic syscalls or unusual /proc behaviour can misbehave — and syscall-heavy workloads pay real overhead, because every syscall now takes a longer path. It occupies a genuine middle rung between container and VM, and which rung you want depends on whether your bottleneck is startup, syscalls, or auditor confidence.

Kata Containers

A container runtime that quietly runs each container (or pod) inside a real lightweight VM. You keep the OCI image format, the Kubernetes integration, the docker-ish workflow; underneath, the workload gets a guest kernel and a hypervisor boundary instead of shared-kernel namespaces.

Where this bites: Kata is a packaging and integration answer, not a different isolation primitive. It uses a VMM underneath — often Firecracker or Cloud Hypervisor. If you are already deep in Kubernetes and want VM-grade isolation without changing how anything is deployed, that is precisely the problem Kata solves. If you are building a product where sandboxes are created by an API call per user request, the container machinery on top is mostly weight you don't need.

Nested virtualization

Running a hypervisor inside a VM. The outer hypervisor has to expose the CPU's virtualization extensions to its guest, so the guest kernel can itself create VMs. Support is a per-platform, per-cloud, per-instance-type question and the answer is frequently no.

Where it bites in practice: developing a microVM platform on a Mac. Apple Silicon has no KVM, so the path is Apple's Virtualization.framework running a Linux VM (via Lima or similar), with nested virtualization enabled so that Linux VM can run KVM and therefore Firecracker. That is how our local development environment works, and it is also why "just run it locally" was three weeks of work rather than an afternoon.

The way people get it wrong: confusing it with running containers inside containers, which is not virtualization at all and has completely different constraints.

Guest and host

The host is the physical machine and its operating system. The guest is the virtual machine running on it and the OS inside that VM. Every term in this glossary is ambiguous until you say which side of that line you mean, and a startling number of confusing bug reports resolve to someone meaning one and being heard as the other.

The concrete example I trip over most: memory. "The VM is using 4 GiB" can mean the guest kernel thinks it has 4 GiB of RAM, or the VMM process on the host has 4 GiB resident. Those numbers diverge enormously — a guest sized at 4 GiB that has touched 300 MiB costs the host roughly 300 MiB, and getting this backwards is how capacity planning goes wrong by a factor of ten.

Boot and lifecycle

Cold boot

An actual boot: the kernel starts at its entry point, initialises subsystems, probes devices, mounts the root filesystem, hands over to an init system, and that init system walks a dependency graph starting services until the machine is usable. For a microVM this is seconds, not the tens of seconds a full PC-emulating VM takes, but it is still the slowest thing in the create path by a wide margin.

Concretely: on our stack, the first spawn of a template — before a snapshot of it exists — is a cold boot of roughly three seconds. Every subsequent create restores the snapshot instead and lands around 179ms p50. Same kernel, same packages, same everything. The only variable is whether the boot already happened once, elsewhere, on someone else's clock.

Snapshot

Here is the single worst overloaded word in this vocabulary, and it is worth slowing down for. In the hypervisor sense, a snapshot is the complete state of a running machine: the contents of guest RAM byte for byte, the register state of every vCPU, and the state of every emulated device. Restoring it does not start a machine — it continues one.

In the cloud-storage sense, a "snapshot" is a point-in-time copy of a disk volume and contains no memory, no CPU state, nothing that was running. Restoring that gives you a disk, which you then have to boot. EBS snapshots, VMware disk snapshots, LVM snapshots, ZFS snapshots — all this second sense.

Snapshot restore

Creating a new VM by loading a machine snapshot rather than booting. A fresh VMM process starts, the memory image is mapped in, device and vCPU state are reinstated, and the guest resumes at the instruction after the pause. Nothing initialises. The kernel does not re-probe devices; it remembers the results. Your interpreter does not re-import its standard library; the objects are already on the heap.

What separates it in practice: restore latency is dominated by how memory is made available, not by how big the snapshot is. Map the memory file privately and the guest pages in lazily as it touches things, so a 4 GiB snapshot does not cost 4 GiB of reading. This is why restore can be tens of milliseconds for a machine that would take seconds to boot.

Checkpoint/restore

The same idea one layer up: freezing a running process (or process tree) to disk and resuming it later. CRIU is the well-known Linux implementation. It captures memory maps, open file descriptors, sockets, timers — everything the kernel holds on the process's behalf.

Compared with a VM snapshot: a process checkpoint has to be restored into a host kernel that will accept it, which makes it fragile across kernel versions, mount layouts, and anything else the process had a handle on. A VM snapshot includes the kernel, so the thing being restored is self-contained. That portability is bought with a larger artifact — you are storing an entire operating system's RAM, not one process's.

Bake

The build-time step that produces the snapshot: boot the template once, wait until it has genuinely finished coming up, pause it, capture the machine state, publish the result. "Baking" is deliberately kitchen-flavoured — you do the slow thing once and then serve portions.

The practical consequence: anything fixed at bake time is fixed forever for every guest restored from that artifact. Guest RAM size is baked. The page size backing that memory is baked. The IP and MAC address the guest believes it has are baked. On PandaStack, an app's memory is governed by the template's baked size rather than the per-app value someone typed into an API request, and the agent overrides the request to match rather than pretending otherwise.

Seed

Our name for the published output of a bake: a directory of artifacts — the guest disk, the device/vCPU state file, the memory image — versioned by generation in object storage, with a pointer file naming the current generation and a checksum manifest. Hosts sync seeds locally so that a create is a local file operation rather than a network one.

Not a standard industry term; other platforms call the equivalent a template, a base snapshot, or a golden snapshot. I include it because it names a real distinction: the template is the recipe, the seed is the specific baked artifact generation you are restoring right now, and knowing which one is stale is most of the operational work.

Fork and clone

Creating a new machine from the current state of a specific running machine, rather than from a template. The child inherits everything the parent had at that instant: loaded model weights, a warmed cache, a half-finished checkout, an authenticated session. Two children forked from one parent start identical and then diverge.

The distinction that matters: this is only cheap if both memory and disk are copy-on-write, and the two halves have very different mechanics. Disk forking on the same host is a metadata-only filesystem clone. Memory forking depends on how the memory image is shared. On our stack a same-host fork lands in the 400–750ms range; across hosts it is 1.2–3.5s, because the snapshot has to travel.

The way people get it wrong: assuming "clone" implies live memory. Most cloud clone operations copy a disk and boot it. If the thing you want is a running process's warm state, ask explicitly whether memory comes along.

Cold start

The user-visible time between asking for compute and that compute doing something useful. It is an end-to-end product measurement, and it includes everything: scheduling and placement, pulling or locating the image, creating the machine, booting or restoring it, starting your runtime, importing your dependencies, and connecting to your database.

Warm pool

A set of instances kept booted and idle so that a request can be handed one immediately. It is the oldest cold-start fix there is, and it works — the machine is already running, so there is no start to be cold.

The bit that matters: a warm pool converts a latency problem into a cost-and-capacity problem. You pay for idle machines, you have to guess the pool depth, you get a cliff when demand exceeds it, and you have to decide how long a used instance may be recycled before you trust it less. The alternative is to make creation fast enough that you don't need a pool — which is why our create path restores a snapshot on demand and there is no pool of idle VMs behind it at all.

Scale-to-zero

Running nothing — literally zero instances, zero cost — when there is no traffic, and creating capacity on the first request. For anything with bursty or long-idle usage this is the difference between an interesting economic model and a boring one.

The distinction that matters: everyone can scale to zero. The only question worth asking is what the wake costs and what survives it. If waking means a cold boot plus a dependency install, you have moved the cost onto your first user rather than removing it. If waking means restoring a snapshot of the already-warm process, the idle saving is real and the wake is short.

Hibernate

Snapshot the running machine, then stop it, keeping the artifacts so it can be resumed later. Borrowed from laptops, and the analogy holds: hibernate writes RAM to disk and powers off, where suspend keeps RAM powered. In platform terms, hibernation is how scale-to-zero preserves state instead of discarding it.

The distinction worth keeping: hibernation is a lifecycle operation, not a storage tier. What you get back is exactly the machine you had, including whatever was mid-flight — an open TCP connection will be broken, a running HTTP server will still be listening, and a program that assumed monotonic wall-clock time will be startled.

# The lifecycle terms above, in the order you actually meet them.
# pip install pandastack ; export PANDASTACK_API_KEY=...
from pandastack import Sandbox

# CREATE = snapshot RESTORE of a BAKED template seed. Not a boot; the boot
# happened once, at bake time, and this is a continuation of it.
sbx = Sandbox.create(template="base", ttl_seconds=600)

# Do the expensive warm-up ONCE in the parent: install deps, load a model,
# clone a repo, prime a cache. This is the state a fork will inherit.
sbx.exec("pip install --quiet pandas")
sbx.filesystem.write("/work/seed.py", "import pandas; print(pandas.__version__)")

# FORK = a new machine from THIS machine's current state, memory included.
# Both children start byte-identical to the parent and then diverge.
a = sbx.fork()
b = sbx.fork()

# Neither child reinstalls anything -- the import is already resident.
print(a.exec("python3 /work/seed.py").stdout)
print(b.exec("python3 /work/seed.py").stdout)

for s in (a, b, sbx):
    s.kill()

Memory and storage

Rootfs

The guest's root filesystem. On the host it is a single file — commonly an ext4 image. Inside the guest it appears as a block device that gets mounted at /. Building one usually means constructing a filesystem tree (often by flattening a container image) and writing it into a formatted image file.

Set against a container image: a container image is a stack of tar layers assembled at runtime by an overlay filesystem, and it has no kernel. A rootfs is one flat filesystem, and the kernel is a separate file the VMM loads directly. This is why microVM platforms tend to build rootfs images from container images rather than consuming container images natively.

Ephemeral vs durable storage

Ephemeral storage lives and dies with the machine — in a microVM, that is the copy-on-write clone of the rootfs, discarded on destroy. Durable storage is a separate volume with an independent lifetime, attached to the machine but outliving it.

This one is not really about data loss, it is about placement. A durable volume is a real file on one specific host, so anything using it is pinned to that host. Ephemeral machines are free to be scheduled anywhere, which is exactly why they can be created in a couple of hundred milliseconds on whichever box has room. Every stateful feature is a negotiation with that constraint.

Copy-on-write (CoW)

Sharing something until somebody writes to it. Two entities point at the same underlying data; reads come straight from the shared copy; the first write to a given block or page triggers a private copy of just that unit, and the writer gets the copy. Nothing is duplicated up front, and only what actually changes costs anything.

The distinction that matters: CoW is a principle, not a mechanism, and it shows up independently at several layers — filesystem blocks, block-device sectors, memory pages, process address spaces. When someone says a platform "uses copy-on-write", the useful follow-up is "for the disk, the memory, or both?", because most systems do one and let you assume the other.

Reflink

A filesystem-level copy-on-write file clone. You ask the filesystem to make a second file that shares the first file's data extents, and it does so by writing metadata only — no data movement, regardless of the file's size. On Linux this is the FICLONE ioctl, available on XFS, Btrfs, and ext4 built with reflink support. The shell version is cp with the --reflink flag.

Compared with a hardlink: a hardlink is one file with two names, so writing through either changes both. A reflink is two independent files that happen to share storage until one is written, at which point the filesystem splits the touched extents. Cloning a multi-gigabyte rootfs takes single-digit milliseconds and consumes essentially no disk until the guest writes something.

dm-snapshot

Copy-on-write at the block layer instead of the filesystem layer. The Linux device-mapper builds a virtual block device from a read-only origin device plus a separate COW device that stores modified blocks. Reads of untouched blocks come from the origin; writes land in the COW store and are remembered by an exception table.

Next to reflink: dm-snapshot works on any block storage, including filesystems that have no reflink support, and it works on whole devices rather than files. The cost is a fixed-size COW device that can fill up — and when a dm-snapshot's COW space is exhausted, the device goes invalid rather than politely returning ENOSPC. Reflink is simpler and has no such cliff, so use it when the filesystem allows and keep dm-snapshot as the fallback.

# What a copy-on-write rootfs clone actually looks like. This is the shape of
# the ~4ms step in a microVM create path -- there is genuinely nothing else to it.

# The template disk, built once, shared by every sandbox on this host.
ls -l --block-size=M /var/lib/pandastack/templates/base/rootfs.ext4
# -rw-r--r-- 1 root root 2048M rootfs.ext4

df --output=used -BM /var/lib/pandastack | tail -1     # note the number

# Clone it. --reflink=always makes the kernel FAIL rather than silently fall
# back to a full byte copy, which is what you want: a silent fallback turns a
# 4ms create into a multi-second one and you will find out in production.
time cp --reflink=always \
  /var/lib/pandastack/templates/base/rootfs.ext4 \
  /var/lib/pandastack/vms/sbx-a1b2/clone.ext4
# real  0m0.004s   <- metadata only; size is irrelevant to the time

df --output=used -BM /var/lib/pandastack | tail -1     # ...essentially unchanged

# Verify the extents really are shared rather than copied:
filefrag -v /var/lib/pandastack/vms/sbx-a1b2/clone.ext4 | grep -c shared

# The guest now writes to its disk. Only the touched extents diverge, and only
# those consume new blocks. Ten sandboxes from one template cost one template
# plus ten deltas -- not ten times two gigabytes.

Page cache

The kernel's in-memory cache of file contents. Read a file, the pages stay in RAM; read it again, no disk involved. It uses whatever memory is otherwise free, which is why "free" memory on a busy Linux box always looks alarmingly low and is not a problem.

The thing to hold onto in virtualization: there are two page caches and they cache the same bytes. The guest kernel caches file data from its virtual disk. The host kernel caches the disk image file that virtual disk lives in. Double caching is the tax you pay for a guest kernel, and it is a real reason microVM density is lower than container density at the same nominal workload.

Ballooning

A cooperative memory-reclaim mechanism. A driver inside the guest allocates pages from the guest kernel and tells the host those pages are not in use, so the host can reclaim the backing memory. Inflating the balloon shrinks what the guest can use; deflating returns it.

The practical consequence: ballooning requires a cooperative, functioning guest. It is not a control the host can simply exercise — if the guest is thrashing, wedged, or hostile, the balloon does not inflate. It is a useful reclaim tool for well-behaved long-lived VMs and a poor foundation for admission decisions on a multi-tenant fleet, which is why we make capacity decisions from observed memory rather than by asking guests to give some back.

Memory overcommit

Promising guests more memory in total than the host physically has, on the correct bet that they will not all touch all of it at once. Linux does this for ordinary processes by default; virtualization stacks do it for guests. It works because a VM sized at 4 GiB that has touched 400 MiB costs the host 400 MiB.

What it comes down to: overcommit is a bet, and the failure mode is not gradual. When the bet loses you are out of physical memory, and what happens next is the kernel OOM killer choosing a victim — quite possibly someone else's tenant. Overcommit without a real signal for how much memory is actually in play is gambling with other people's workloads.

Working set

The memory a workload actually touches over some window, as opposed to the memory it was told it could have. A guest configured with 4 GiB whose steady state is 600 MiB has a 600 MiB working set, and that is the number that determines how many of them fit on a host.

This is the term that changed how we schedule. Counting committed memory — the number in the create request — caps a 32 GiB host at around seven 4 GiB sandboxes while the great majority of RAM sits idle. Admitting on measured working set instead, with a per-sandbox reserve for headroom, fits far more real work on the same hardware. The switch is a per-host mode on our agents precisely because getting it wrong is an OOM kill rather than a slowdown.

UFFD / userfaultfd

A Linux kernel API that lets a userspace program handle page faults for a region of memory. You register the region, and when something touches a page that is not present, the kernel notifies your handler over a file descriptor instead of resolving the fault itself. Your handler supplies the page — with UFFDIO_COPY for real content, or UFFDIO_ZEROPAGE for a page that should be zeros — and the faulting thread resumes.

Why it matters for microVMs: it turns snapshot memory into a demand-paged resource. Instead of a multi-gigabyte memory image having to be present locally before the guest can resume, the guest resumes immediately and each page arrives when first touched. On PandaStack that handler fetches 4 MiB chunks by HTTP range request from object storage, with a baked header recording which chunks are entirely zeros so those are filled without any fetch at all.

The way people get it wrong: assuming UFFD streams the disk too. It does not. Memory is demand-paged; the rootfs still has to be a local file, because copy-on-write cloning needs local storage. Streaming removes the memory download, not the disk one.

Hugepages

Memory mapped in 2 MiB units instead of the usual 4 KiB. Fewer pages means fewer page-table entries, fewer TLB misses, and — for a demand-paged restore — 512 times fewer faults to service, because one fault covers 2 MiB of guest memory.

The distinction that matters, and it surprised us: hugepage backing is a property of the snapshot, not a runtime flag. A guest booted on hugepages produces a snapshot that can only be restored through a userfaultfd backend — the ordinary file-backed restore path rejects it. Turning the flag on therefore does nothing for existing snapshots; they stay 4 KiB until the template is re-baked. Anything fixed at bake time stays fixed, and this is the least obvious instance of that rule.

Networking

TAP device

A virtual network interface that carries Ethernet frames between the kernel and a userspace program. The VMM opens the TAP device and presents the guest with a virtual NIC; frames the guest transmits appear as reads for the VMM, which writes them to the TAP, and the host networking stack takes it from there.

Compared with TUN: TAP is layer 2 and carries Ethernet frames, TUN is layer 3 and carries IP packets. VMs want TAP, because the guest believes it has a real NIC with a MAC address and expects ARP to work.

Network namespace

An isolated copy of the Linux network stack: its own interfaces, its own routing table, its own firewall rules, its own socket table. A process inside one sees only those. It is a kernel primitive, not a container feature — containers use it, but you can create one with a single ip command and nothing else involved.

Why microVM platforms lean on it: putting each guest's TAP device in its own namespace means every sandbox can be handed identical network configuration without collisions, and teardown is one operation that removes the interfaces, the routes, and the rules together. No leaked iptables entries, no orphaned interfaces accumulating over a week of churn.

veth pair

Two virtual Ethernet interfaces joined back to back: whatever goes in one end comes out the other. Put one end in a namespace and leave the other in the host, and you have a cable between them. It is the standard way to connect an isolated namespace to the outside world.

NAT

Rewriting addresses on packets as they cross a boundary, so machines with private addresses can reach the internet through a shared public one. For sandbox fleets it is what lets thousands of guests hold private addresses out of a reserved range without any of them needing routable addresses of their own.

The operational catch: NAT is stateful. Every connection occupies an entry in the host's connection-tracking table, and that table has a finite size. A fleet of enthusiastically outbound sandboxes will exhaust it, and the symptom is not a clean error but connections that mysteriously fail to establish.

NATID (our version of all of the above)

The concrete assembly, so the abstract terms land. Each sandbox gets a /30 out of a private /16 — 16,384 of them per host, which is the hard ceiling on sandboxes per agent even though memory binds long before that. Each slot is a namespace, a veth pair with one end in the host and one inside, a TAP device inside for Firecracker, and the NAT rules to match.

The part worth stealing: building that from scratch takes around 100ms, which would dominate a 179ms create. So the slots are pre-created before anyone asks for them, and allocating one at create time is roughly 9ms of patching a MAC address. When the pre-built pool drains, allocation falls back to building a slot on demand — slower, but it does not fail.

# A per-sandbox network namespace, from nothing, in six commands. This is the
# work that gets pre-built so the create path doesn't have to pay for it.
ID=a1b2
SUBNET=10.200.4          # each sandbox gets a /30: .0 network, .1 host, .2 guest

ip netns add ns-$ID                                   # the isolated stack

# veth pair: vh-* stays in the host, vg-* goes into the namespace.
ip link add vh-$ID type veth peer name vg-$ID
ip link set vg-$ID netns ns-$ID
ip addr add $SUBNET.1/30 dev vh-$ID && ip link set vh-$ID up

# The TAP device Firecracker will attach the guest NIC to, inside the namespace.
ip netns exec ns-$ID ip tuntap add tap0 mode tap
ip netns exec ns-$ID ip addr add $SUBNET.2/30 dev tap0
ip netns exec ns-$ID ip link set tap0 up

# NAT so the guest can reach the internet through the host's address.
iptables -t nat -A POSTROUTING -s $SUBNET.0/30 -j MASQUERADE

# Teardown is ONE command, and it takes the interfaces, routes and rules with
# it. That atomicity is the reason to use a namespace per sandbox rather than
# a shared bridge -- no leaked rules after a week of churn.
ip netns del ns-$ID

Egress

Traffic leaving your infrastructure. It matters on a sandbox platform for two reasons that have nothing to do with each other: it is the network cost that scales with what tenants do rather than what they reserve, and it is the direction abuse travels. Untrusted code with unrestricted outbound access is a proxy, a scanner, and a mining client waiting to happen.

vsock

A socket family for host-to-guest communication that does not use the network at all. Endpoints are addressed by a context ID and a port rather than an IP address, and the transport is the hypervisor. No interfaces, no routes, no firewall, no DNS.

The bit that matters: vsock is a control channel, not networking. It works even when the guest has no network configured or its networking is deliberately restricted, which makes it the right pipe for agent-to-guest control traffic — readiness probes, lifecycle commands, telemetry. It is not a substitute for the guest's own network access.

Operations and economics

Multi-tenancy

More than one customer's workload on the same physical hardware. It is the reason cloud economics work, and the reason every isolation decision in this glossary is load-bearing. The alternative — one customer per machine — is simple, secure, and priced accordingly.

Worth separating: isolation of security (can tenant A read tenant B's data?) from isolation of performance (can tenant A make tenant B slow?). Different mechanisms solve them. Hardware virtualization gives you the first and does surprisingly little for the second.

Noisy neighbour

A tenant whose resource use degrades a co-located tenant's performance without any security boundary being crossed. CPU is the obvious channel; memory bandwidth, last-level cache, disk IOPS, and the connection-tracking table are the ones that actually get you, because the first is easy to cap and the rest are shared in ways that are hard to partition.

What actually matters: a strong security boundary does not imply a strong performance boundary. Two microVMs are cryptographically irrelevant to each other and can still fight over L3 cache. Fairness needs scheduler weights, I/O throttling, and admission limits — separate machinery from the isolation boundary.

Density

How many workloads fit on one host. It is the number that determines your cost per sandbox and therefore your price, and it is almost always bounded by memory rather than CPU, because CPU can be timeshared and RAM cannot be conjured.

The distinction that matters: density measured against configured sizes and density measured against real usage differ by an enormous factor, and only the second one is real. Everything in the memory section of this glossary — working set, overcommit, CoW, demand paging, hugepages — exists to close the gap between those two numbers.

Admission control

Deciding, before you accept a workload, whether you actually have room for it. The system says no now, deliberately, instead of saying yes and failing later in a way that harms workloads that were already running.

Set against a quota: a quota is a per-tenant policy about how much they are entitled to. Admission control is a per-host physics question about whether the machine can take this right now. You need both, and they fail differently: quota exhaustion should be a clear 4xx that names the limit, while capacity exhaustion should be a retry — which is why a wake or a deploy that gets refused for capacity on our side parks and retries with backoff rather than being marked failed.

Oversubscription

Selling more capacity than you physically have, in aggregate, because customers do not all use their allocation simultaneously. Airlines do it with seats; every cloud provider does it with CPU, and most do it with memory.

The distinction versus overcommit: overcommit is the mechanism (the kernel handing out address space it cannot fully back), oversubscription is the business decision (the ratio you are willing to run at). People use the words interchangeably, and the difference matters when you are deciding who owns the risk — an engineer tunes the mechanism, but the ratio is a commercial choice about how much degradation you will tolerate at the peak.

vCPU-hour and GiB-hour

The two standard units for billing compute: one virtual CPU for one hour, and one gibibyte of memory for one hour. Nearly every sandbox platform prices in some combination of them, which is what makes cross-vendor comparison possible at all.

The distinction that matters, and it is where comparisons go wrong: whether you are billed for what you reserved or what you used. Memory is genuinely reserved — a guest sized at 4 GiB denies that RAM to everyone else whether it touches it or not — so committed GiB-hours are honest. CPU is timeshared, so billing reserved vCPUs for an idle sandbox charges for nothing. We bill memory on committed GiB-hours and CPU on active CPU-seconds actually burned, at $0.054 per vCPU-hour and $0.0162 per GiB-hour, and the reason to split them that way is the physics, not the marketing.

Serverless

A billing and operations model: you do not provision or manage machines, you are charged for execution rather than for uptime, and capacity scales without you asking. That is the complete definition, and every word of it is about economics and operations.

The six confusions worth memorising

If you keep nothing else from this, keep these. Each one has cost me or someone I was talking to at least an afternoon.

1. "Snapshot" means two unrelated things. A disk snapshot is a copy of a volume and still has to boot. A machine snapshot is RAM plus vCPU registers plus device state and resumes without booting. Ask which one before you believe any latency claim.
2. "Sandbox" names the goal, not the boundary. A language-level restriction, a shared-kernel container, and a hardware-isolated VM are all sandboxes, and they differ by orders of magnitude in what an attacker must break.
3. Cold start is not cold boot. Cold boot is one ingredient of cold start, often not the biggest. Measure what your user waits for, end to end.
4. A container is a process, not a machine. It shares your kernel. Every property people find surprising about containers follows from that one fact.
5. "Serverless" describes billing, not isolation. It is orthogonal to who your code is co-tenanted with, and vendors are not eager to clarify.
6. Overcommit is a mechanism; oversubscription is a ratio you chose. Confusing them means nobody owns the decision about how much risk you are running at peak.

The summary

Most of the confusion in this space is not conceptual, it is lexical. The concepts are reasonable and mostly composable: isolate at some boundary, avoid booting twice, share what has not been written to, and count the memory that is actually touched. The trouble is that the industry assigned two or three meanings to half the nouns, then built pricing pages on top of the ambiguity.

The habit that helps most is asking one question per term: where is the boundary, which artifact are we talking about, and is that number reserved or used. Nearly every misunderstanding above collapses under one of those three.

If a definition here is wrong or a term is missing, tell me and I will fix the entry. This is meant to be the page I hand to a new engineer on day one, and it only earns that by being accurate.