Tidal: Turning the Host Into a Cache, and Paying for the Seconds You Touch

Every VM platform bills the same way for the same historical reason: a virtual machine was once a slice of a physical one, so you paid for the slice — so many vCPUs, so much RAM, per hour, whether you touched them or not. That model made sense when the hypervisor genuinely had to set the capacity aside. On a Firecracker fleet where a sandbox restores from snapshot in 179ms at p50, it stops making sense: the expensive thing is no longer holding capacity, it's touching it. So we rebuilt our compute layer around a different mental model — the host is a cache — and changed billing to match. You pay for the CPU-seconds you actually burn and the memory you actually keep resident, not the capacity you committed. This post is the engineering behind that: burstable vCPUs with kernel fair-share, active-CPU and working-set metering, and the pressure ladder that lets us overcommit hosts without ever OOM-killing a guest.

The host is a cache

A CPU cache doesn't reserve a line for every byte of RAM that might someday be accessed; it keeps the hot set close, pulls cold lines on demand, and the whole machine is built on the bet that the hot set is small. A host full of microVMs has exactly that shape. Most sandboxes are idle most of the time — waiting on a user, an LLM, a webhook — with a large committed allocation and a small hot working set. Treat committed vCPUs and committed RAM as the address space, and the host's real job becomes keeping every guest's hot set resident and fast while the cold majority costs approximately nothing.

Two properties of our substrate make this safe in a way it isn't for a general-purpose cloud. First, snapshot-restore is our native lifecycle: evicting a fully idle guest is not an outage, it's a state transition we already perform on every create, with a sub-second path back. Second, cgroup v2 gives us per-VM accounting and per-VM reclaim as first-class kernel primitives — we can see exactly what each guest burns and touches, and we can shrink exactly one guest without perturbing its neighbors. Cache semantics need cheap eviction and precise accounting; we happen to have both.

Eight burstable vCPUs, fair-shared by the kernel

The first user-visible change: every first-party template now boots with 8 vCPUs. Not as a tier you pick and pay for — as the default shape of every sandbox and app. On an idle host, one sandbox can genuinely saturate all 8 cores; an npm install or a Next.js build gets the whole machine's parallelism instead of a small fixed ceiling. Under contention, the cgroup CPU controller fair-shares the physical cores across whoever is burning.

Fair-share is the part people distrust, so we measured it in the worst configuration we ship: three VMs, each spinning all their vCPUs flat out, on a host overcommitted 3x. The kernel split the metal dead even — 119, 120, and 119 CPU-seconds respectively over the measurement window, with zero starvation. No priority inversion, no lucky VM. That symmetry is what makes 'burstable' honest: burst capacity is free headroom when the host is quiet and an equal share when it isn't, and there is no configuration in which a neighbor's burst starves you.

The larger default guest did not slow the boot path, which was the thing we worried about most. Restores of the new base template measured 69ms at p50 on production (without hugepages), against the 179ms p50 end-to-end create — the snapshot path is indifferent to how big a guest it's rehydrating.

Billing active CPU-seconds

CPU billing now meters active CPU-seconds: the cgroup's usage_usec, read from cpu.stat and integrated over the VM's life. This is the number the kernel already keeps in order to schedule fairly; we just made it the number on the invoice. If your process burns a core for a second, you pay for one CPU-second. If it blocks on a socket, you pay nothing for that gap — no matter how many vCPUs the guest has.

# The meters are the kernel's own accounting, one cgroup per VM:
cat /sys/fs/cgroup/pandastack/<vm-id>/cpu.stat        # usage_usec -> active CPU-seconds
cat /sys/fs/cgroup/pandastack/<vm-id>/memory.current  # resident bytes -> working-set GiB-seconds

The measured gap between this and committed-basis billing is not subtle. A VM running a sustained real workload was billed 160 CPU-seconds over a window where the committed basis — every vCPU times wall clock — would have charged 1240. An idle-but-running VM accrued 2 CPU-seconds across 306 seconds of wall clock: the cost of timers, heartbeats, and a guest agent, which is to say the cost of existing. Under committed billing those two VMs cost the same. Under active billing, the idle one costs approximately nothing, which is what it consumed.

The rate card follows the meters. Sandboxes: $0.054 per vCPU-hour of active CPU, plus $0.0162 per GiB-hour of working-set memory. Apps: from about $23/month — the 4 GiB always-on memory floor at $0.008/GiB-hour — plus $0.004 per vCPU-hour of active CPU on top. The 8 vCPUs themselves carry no reservation price; you pay for the seconds you touch them.

Working-set RAM billing

Memory billing meters the working set: resident bytes, integrated into GiB-seconds, clamped to the committed size. The clamp means the new meter can only read at or below the old committed basis — you can never pay for more than you asked for, and you pay less whenever your resident set is smaller, which for most guests is most of the time. A sandbox that commits a large allocation but keeps a small resident set is billed on the resident set.

Working-set billing has a subtle honesty requirement: resident memory is partly under the platform's control, not just the guest's. If we never reclaimed anything, every guest's residency would ratchet up to its high-water mark — page cache, dead heap, freed-but-never-returned allocator pages — and 'working set' would quietly converge back to 'committed'. So the reclaim machinery described next isn't only host self-defense. It's the mechanism that keeps the meter honest: memory your workload hasn't touched in a while gets compressed or dropped, leaves the resident set, and stops costing you.

The pressure ladder

Overcommitting RAM across guests requires an answer to the obvious question: what happens when everyone's working set grows at once? Our answer is a ladder of escalating, bounded responses. The design rule is that each rung must be invisible or near-invisible to the guest it touches, and the final rung must never be an OOM kill.

• Rung 0 — do nothing. Free memory is wasted memory; when the host has headroom, working sets grow untouched and page cache stays hot. Most hosts sit here most of the time.
• Rung 1 — squeeze. Under pressure, we drive idle VMs down toward 70% of their peak residency with targeted memory.reclaim. The cold tail lands in zswap, which compresses the sparse, mostly-zero pages typical of an idle guest at roughly 50x — so 'reclaimed' mostly means 'compressed in RAM', not 'sent to disk'. Touching a squeezed page decompresses it on fault; the guest sees a little latency, never a failure.
• Rung 2 — freeze. Only under absolute scarcity, we Thaw-freeze the coldest idle sandbox: snapshot its full state, release everything, and rely on the sub-second restore path to bring it back on next touch. This is eviction in the cache sense — the guest's state is safe, its residency is zero.

We rehearsed the worst case rather than waiting for it: a simulated memory storm injecting a 17.4 GiB burn into a busy host. The ladder walked its rungs and the host recovered from 3.9 GB available to 21.3 GB available in about 70 seconds — with zero VM kills. That last number is the design goal stated as a result: guests on the burstable class can be slowed and squeezed under host pressure, but never destroyed by it.

The idle reclaim cycle

The ladder handles storms; a quieter loop handles the steady state. A background cycle watches per-VM activity and runs memory.reclaim against guests that have gone cold, continuously converting idle residency back into burst headroom. Measured on production: one idle VM's residency dropped from 3258 MB to 897 MB in 12.9 seconds, and a routine automated cycle reclaimed 972 MB in 4.8 seconds. Those megabytes stop being billed to the idle guest and become the headroom a neighbor's 8-core build spike lands in. That's the cache loop closed: cold state compresses and stops costing, hot work gets the physical machine.

What we deliberately didn't do

Databases are excluded from all of this, by design. Managed Postgres is our guaranteed class: a database pays its full reservation (1g at $10.95/month, 4g at $37.23/month, 16g at $142.35/month), is never overcommitted, never squeezed, and never frozen by the ladder. Postgres treats RAM as its own cache — shared buffers, plus the OS page cache over the heap files — and a platform reclaiming that memory doesn't save anything real; it converts host pressure directly into query latency. Two cache layers fighting over the same pages is a well-known pathology, so we didn't build it. Databases do scale to zero on their own terms — auto-suspend after 15 minutes idle, wake on connect — but that's a clean state transition, not a squeeze.

Admission control hasn't flipped yet, either. The scheduler still admits VMs against committed vCPU and RAM, not measured working sets — so a host's packing density is currently bounded by the conservative number even though billing is metered on the honest one. That ordering is deliberate: bill on the honest meter first, prove the reclaim machinery under real storms, and only then let admission trust working sets. Until that flip, the density upside of the cache model is real but bounded.

And burstable means burstable: 8 vCPUs is the shape of the guest, not a reservation of 8 physical cores. Under contention you get the fair share we measured above, not the whole machine. Workloads that need capacity held for them regardless of neighbors belong in a guaranteed class — today that's databases. That's the honest boundary of this design.

The summary

The bill now has the same shape as the work. A sandbox that thinks hard for a burst and then waits pays for the burst; an app that serves traffic all day pays for the CPU-seconds it burned serving it; memory that goes cold compresses, stops being billed, and turns into a neighbor's burst headroom. The host runs like a cache — hot set resident and fast, cold set compressed or evicted behind a sub-second restore — and the pressure ladder means overcommit degrades to latency, never to a dead VM. Pay for the seconds you touch; the machine handles the rest.