Isolating Customer-Managed Key Operations in microVMs

Somewhere in every enterprise sales cycle a security reviewer asks whether the customer can bring their own encryption key. You say yes, because the deal is worth more than the sprint, and six weeks later you ship BYOK. The marketing page says the customer controls their key. What the architecture says is that your service now performs cryptographic operations with material belonging to one named tenant, inside a process that is simultaneously serving everyone else.

I'm Ajay; I build PandaStack, a Firecracker microVM platform, so read this as opinionated. The question I want to sit with is the one a good reviewer asks second, after the diagram: when your process decrypts tenant A's data, what exactly stops tenant B's request — same process, same host, same heap, one page table — from seeing that plaintext or that key? On most implementations the honest answer is "our code does not do that on purpose," which is a statement about intent, not about isolation.

What BYOK actually means, without the hand-waving

Nobody encrypts a hundred million objects directly with the customer's KEK. KMS and HSM APIs are rate-limited, latency-bound and usually payload-limited to a few kilobytes, and you do not want a network round trip per byte. So essentially every real implementation is an envelope scheme: generate a fresh DEK per object or per batch, encrypt the data locally with the DEK, then ask the customer's KMS to wrap that DEK under their KEK. You store ciphertext plus wrapped DEK. You do not store the DEK.

1. Write path: generate a random 256-bit DEK in the worker, AES-GCM the object with it, call the customer's KMS Encrypt/Wrap on the 32-byte DEK, persist ciphertext + wrapped DEK + key ARN + a nonce. Zero the DEK.
2. Read path: fetch ciphertext + wrapped DEK, call the customer's KMS Decrypt/Unwrap, receive the plaintext DEK over TLS, decrypt the object, do the work, zero the DEK.
3. Revocation path — the one the customer actually bought: they disable or delete the KEK in their own KMS. Every wrapped DEK you hold becomes undecryptable. Your copy of their data turns into noise without you doing anything, on their schedule, without a support ticket.

That last item is the honest value proposition, and it is a good one. BYOK is a kill switch the customer holds, plus an audit trail in their KMS showing every time you asked to unwrap something. The strongest designs go further: the KEK never leaves the customer's KMS or HSM at all, you only ever send it wrapped DEKs, and if the customer imported their own key material with no export permitted, then in a meaningful sense you never had it.

But be precise about where the risk went, because a lot of BYOK documentation is quietly misleading here. BYOK does not mean the customer's data is never in your RAM. It means the customer can decide when you stop being able to put it there. The unwrapped DEK and the object plaintext unavoidably touch your infrastructure — that is the entire point of the read path — and the interesting security question is not about the KEK you never had. It is about the thirty milliseconds during which you had everything.

BYOK moves the key out of your database and into the customer's KMS. It does not move the plaintext out of your heap. The plaintext is where it always was: in the process doing the work.

The shared-process problem

The default architecture is one long-lived service that handles crypto for every tenant. It is the default because it is obviously correct on every axis except this one: connection pools stay warm, KMS clients are reused, and the DEK cache — you will build a DEK cache, everyone does, because KMS calls cost real money and real latency — amortises unwraps across requests. Which means that after a week of uptime, the resident memory of that process contains unwrapped data keys for a substantial fraction of your customer base.

Now enumerate what reads that memory. Every heap-disclosure bug in every dependency in that process. Every crash dump your error reporter uploads to a third-party SaaS, helpfully including local variables. Every core file the kernel writes to /var/crash when a native extension segfaults, which your log-shipper then compresses and forwards. Every profiler heap snapshot an engineer takes during an incident and pastes into a shared channel. Every "let's add a debug endpoint that dumps the cache, we'll remove it before launch." Every time the process swaps and those pages land unencrypted on a disk you do not shred.

None of those are exploits. They are ordinary operational events that happen to be, in this configuration, cross-tenant data disclosures. A heap dump is a remarkably efficient way to violate a data-processing agreement: one file, every tenant, no attacker required, uploaded by your own observability stack with the best of intentions.

The garbage-collected runtimes make it worse rather than better. A moving collector may copy your key buffer to a new location and leave the old bytes intact until that region is reused. Strings get interned. A logging middleware that serialises request context for a trace does not know which field is a key. And the lifetime of anything in a managed heap is decided by the collector, not by you — which matters enormously in the next section but also right now, because "we clear the cache entry" and "those bytes are gone" are not the same sentence.

The shape: one disposable machine per key operation

The alternative is structurally boring, which is usually a good sign. Give each tenant crypto session — or each bulk operation, if per-request is too fine-grained for your latency budget — its own virtual machine. A Firecracker guest, its own kernel under KVM, its own page tables, its own memory that no other tenant's code is running inside. The unwrapped DEK exists only in that guest. When the operation completes, you destroy the machine.

The security property worth the trouble is one sentence: the blast radius of a memory-disclosure bug becomes one tenant's session rather than the fleet's key cache. A heap bug in your indexer still leaks a DEK — it leaks the DEK belonging to the tenant whose data that guest was already processing, to code that was already inside that guest. That is not nothing, but it is a single-tenant incident with a bounded time window, and you can name the tenant and the object set in the notification without saying "we are still determining scope."

The mechanics that make it affordable: every create is a restore of a baked snapshot rather than a cold boot, which on PandaStack is about 179ms p50 and 203ms p99 end to end. The one-time cold boot of a template is around 3 seconds and you pay it once. A crypto session that lasts two seconds and costs 179ms to stand up is a tax you can actually afford per operation, which is the only reason this design is not purely theoretical.

// envelope.go -- the read path, written so the dangerous window is visible.
//
// Everything interesting in this file happens between line "dek, err :=
// kms.Unwrap(...)" and line "zero(dek)". Outside that window the DEK does
// not exist in this address space. Inside it, it exists in a process that
// -- in the shared-service design -- is also serving other tenants.

package crypto

import (
	"crypto/aes"
	"crypto/cipher"
	"crypto/rand"
	"errors"
	"runtime"
)

type Envelope struct {
	KeyARN     string // the CUSTOMER's KEK. We never hold this key.
	WrappedDEK []byte // 32-byte DEK, encrypted under their KEK.
	Nonce      []byte // 12 bytes, unique per (DEK, message). See the RNG note.
	Ciphertext []byte
}

// KMS is the customer's key service. Unwrap is a network call to THEIR
// account: it is audited on their side, rate-limited on their side, and
// revocable on their side. That is the whole product feature.
type KMS interface {
	Unwrap(keyARN string, wrapped []byte, aad map[string]string) ([]byte, error)
	Wrap(keyARN string, dek []byte, aad map[string]string) ([]byte, error)
}

func Open(k KMS, tenantID string, e Envelope) ([]byte, error) {
	// AAD binds the wrapped DEK to this tenant and this object. Without it,
	// a wrapped DEK lifted from tenant A's row and pasted into tenant B's
	// row still unwraps -- the KEK does not know which row it came from.
	aad := map[string]string{"tenant": tenantID, "arn": e.KeyARN}

	dek, err := k.Unwrap(e.KeyARN, e.WrappedDEK, aad) // <-- window opens
	if err != nil {
		return nil, err
	}
	defer zero(dek) // <-- window closes, for a given value of "closes"

	if len(dek) != 32 {
		return nil, errors.New("unexpected DEK length")
	}

	block, err := aes.NewCipher(dek)
	if err != nil {
		return nil, err
	}
	aead, err := cipher.NewGCM(block)
	if err != nil {
		return nil, err
	}
	return aead.Open(nil, e.Nonce, e.Ciphertext, []byte(tenantID))
}

func Seal(k KMS, tenantID, keyARN string, plaintext []byte) (Envelope, error) {
	dek := make([]byte, 32)
	if _, err := rand.Read(dek); err != nil { // crypto/rand -> getrandom(2)
		return Envelope{}, err
	}
	defer zero(dek)

	nonce := make([]byte, 12)
	if _, err := rand.Read(nonce); err != nil {
		return Envelope{}, err
	}
	// GCM nonce reuse under the same key is not "weaker". It reveals the
	// XOR of two plaintexts AND leaks the GHASH authentication subkey,
	// which lets an attacker forge. Read the snapshot section before you
	// let a restored VM anywhere near this line.

	block, _ := aes.NewCipher(dek)
	aead, _ := cipher.NewGCM(block)
	ct := aead.Seal(nil, nonce, plaintext, []byte(tenantID))

	wrapped, err := k.Wrap(keyARN, dek, map[string]string{
		"tenant": tenantID, "arn": keyARN,
	})
	if err != nil {
		return Envelope{}, err
	}
	return Envelope{keyARN, wrapped, nonce, ct}, nil
}

// zero is the honest version: it does what it says on THIS compiler today,
// and it makes no promise about copies the runtime made behind your back
// during a stack growth, an interface boxing, or a GC cycle.
func zero(b []byte) {
	for i := range b {
		b[i] = 0
	}
	runtime.KeepAlive(b) // stop the write loop being elided as dead
}

Read that file as an argument rather than a library. The window is short and clearly marked, and in a single-tenant process that would be close to the end of the story. In a shared multi-tenant process, "short" is measured against a process lifetime of weeks and a heap shared with every request you serve. The microVM version does not shorten the window. It shrinks the room the window is in.

import json

from pandastack import Sandbox

# One guest per crypto session. It exists for as long as the operation and
# not one second longer, and nothing else runs inside it.
#
# What goes IN:  the wrapped DEK (useless without the customer's KEK) and a
#                short-lived credential scoped to Decrypt on exactly that key.
# What NEVER goes in: your platform's long-lived cloud role, your KMS admin
#                credential, or another tenant's anything.

CRYPTO_TEMPLATE = "base"          # baked from a guest that never held a key
SESSION_BUDGET_SECONDS = 120


def index_encrypted_object(tenant_id: str, envelope: dict, object_uri: str) -> dict:
    """Decrypt one tenant's object and build a search index from it.

    This is the case where a microVM is the right tool: we genuinely need
    the plaintext. If the operation could be pushed into the customer's KMS,
    it should be -- see the "when this is the wrong tool" section.
    """
    sbx = Sandbox.create(
        template=CRYPTO_TEMPLATE,
        ttl_seconds=SESSION_BUDGET_SECONDS + 60,   # platform backstop
        metadata={"tenant": tenant_id, "purpose": "cmk-index", "snapshot": "never"},
    )

    try:
        # Reseed the guest RNG BEFORE any crypto happens. A guest restored
        # from a snapshot resumes with the RNG state frozen at bake time,
        # and every guest restored from the same snapshot resumes with the
        # SAME state. See the snapshot section -- this line is not optional.
        sbx.exec("/usr/local/bin/reseed-entropy && systemctl restart chrony")

        sbx.filesystem.write("/run/session/envelope.json", json.dumps(envelope))
        sbx.filesystem.write("/run/session/kms.token", mint_scoped_kms_token(
            tenant_id=tenant_id,
            key_arn=envelope["key_arn"],
            actions=["kms:Decrypt"],     # not Encrypt, not DescribeKey, not *
            ttl_seconds=SESSION_BUDGET_SECONDS,
        ))

        # The worker unwraps, decrypts, tokenises, emits an index shard, and
        # zeroes what it can. /run/session is a tmpfs: never on the rootfs,
        # never in a reflink clone, never in a disk image somebody archives.
        r = sbx.exec(
            f"cmk-worker --envelope /run/session/envelope.json "
            f"--source {object_uri} --out /run/session/shard.bin",
            timeout_seconds=SESSION_BUDGET_SECONDS,
        )
        if r.exit_code != 0:
            raise RuntimeError(r.stderr[-2000:])

        # The index shard leaves; the DEK and the plaintext do not.
        shard = sbx.filesystem.read("/run/session/shard.bin")
        return {"tenant": tenant_id, "shard": shard, "kms_request_id": r.stdout.strip()}

    finally:
        # This is the real zeroization primitive. Not a memset the compiler
        # might elide, not a hope about what the GC did -- the machine that
        # held the key stops existing, kernel and page tables included.
        sbx.destroy()

# DO NOT, in this code path, call sbx.snapshot() or sbx.fork(), and do not
# let scale-to-zero auto-hibernate touch this sandbox. All three write guest
# RAM -- including the DEK -- somewhere it outlives the session.

The snapshot warning, which is the sharpest point in this post

This trap is specific to the exact technology I am recommending, which is why it goes near the top rather than in a footnote. Snapshot-restore is the thing that makes per-operation microVMs affordable — it is why a create costs 179ms instead of 3 seconds — and it is a memory dump with a friendly name. On PandaStack, snapshots are not even local-only: they are published to object storage and streamed back to whichever host needs them, page by page, over UFFD. A DEK in a snapshot is a DEK in a bucket.

Fork has the same problem with an extra multiplier. Forking is copy-on-write on guest memory, which is exactly why it is fast — and it means a fork of a key-holding guest hands that key to every child. Five forks, five copies of one tenant's DEK, in five machines that may go on to process a different tenant's request if your dispatcher is careless. Fork is a wonderful primitive for branching a build or a test matrix. It is a key-duplication primitive for this workload.

1. Bake templates only from guests that have never held key material. The template is baked once, on a build host, doing nothing but installing the worker binary. If the bake process ever unwraps a real DEK to "test the path," that template is burned and must be rebuilt.
2. Never snapshot a live crypto session. Not for debugging, not to reproduce a bug, not because a customer asked you to capture state. The reproduction you want is a fresh guest plus a synthetic key, and if you cannot reproduce it that way you have learned something useful about your logging.
3. Turn off automatic hibernation for these sandboxes. Scale-to-zero auto-hibernate is a feature I like and ship, and it works by taking a snapshot. On a crypto session it is a silent key-persistence mechanism with a cost-saving justification, which is the worst kind of footgun. Short TTLs instead: a session that is idle is a session that should be destroyed, not frozen.
4. Keep key material off the rootfs too. Use a tmpfs mount for anything session-scoped. The rootfs is CoW-cloned by reflink and may be captured alongside a snapshot; a tmpfs page is guest RAM that only ever existed in a machine you are about to delete.
5. Make it structural, not a code review rule. Tag the sandbox at creation, and have the snapshot and fork paths refuse a sandbox carrying that tag. A policy that depends on nobody being tired at 5pm on a Friday is not a policy.

The related trap: a restored guest wakes up with the same randomness

This one is subtler and disqualifying in a different way. A snapshot freezes the entire guest, and "the entire guest" includes the kernel's random pool and any userspace CSPRNG state that was already seeded. Restore that snapshot ten thousand times and you have ten thousand machines whose next bytes from the random pool are identical. For a build runner that is a curiosity. For anything generating DEKs, IVs or GCM nonces it is catastrophic — and it is catastrophic silently, because the output still looks random to every test you would think to write.

AES-GCM is the sharp edge here. Reusing a nonce under the same key does not merely weaken confidentiality; it exposes the XOR of the two plaintexts and leaks the GHASH authentication subkey, which converts an eavesdropper into a forger. If two restored guests hold the same key and draw the same nonce, you have handed that away for free.

So: reseed before any crypto happens on a restored guest, and treat that as a precondition of the workload rather than a nice-to-have. Draw fresh entropy from the host through virtio-rng, push it into the kernel pool, and only then let the worker start. Firecracker exposes an entropy device precisely because snapshot-restore creates this problem; the same guest kernel that made your boot fast is the one carrying a stale pool. And check the same thing for anything else frozen in time — a restored guest also believes it is the moment of the snapshot, which is its own problem the first time it validates a TLS certificate.

Zeroization, and why it is weaker than it sounds

The textbook answer to key material in memory is a short list: lock the pages so they never reach swap, overwrite them the instant you are done, and use a primitive the compiler is not allowed to optimise away. All three are worth doing. None of them is as strong as the security section of your architecture doc implies.

// The textbook version, in the language where you can most nearly do it.
package keybuf

import (
	"crypto/subtle"
	"runtime"

	"golang.org/x/sys/unix"
)

// Locked is a key buffer that (a) never reaches swap and (b) is overwritten
// on Close. Both of those sentences carry asterisks. See below.
type Locked struct{ b []byte }

func New(n int) (*Locked, error) {
	// Page-aligned anonymous mapping. Not from the Go heap, so the garbage
	// collector will not relocate it behind our back mid-operation.
	b, err := unix.Mmap(-1, 0, roundUpToPage(n),
		unix.PROT_READ|unix.PROT_WRITE,
		unix.MAP_PRIVATE|unix.MAP_ANONYMOUS)
	if err != nil {
		return nil, err
	}
	// mlock: keep these pages resident so the key is never written to swap.
	// Subject to RLIMIT_MEMLOCK, which on a default host is small enough
	// that this call fails in production and succeeds on your laptop.
	if err := unix.Mlock(b); err != nil {
		_ = unix.Munmap(b)
		return nil, err
	}
	// Do not include this region in a core dump. Without it, one segfault
	// writes the key to /var/crash and your log shipper does the rest.
	_ = unix.Madvise(b, unix.MADV_DONTDUMP)
	return &Locked{b: b[:n]}, nil
}

func (l *Locked) Close() {
	// In C this is explicit_bzero(3) or memset_s: standardised precisely
	// because a compiler is entitled to delete a memset to memory that is
	// never read again, and every compiler eventually does.
	//
	// Go has no explicit_bzero. subtle.ConstantTimeCopy over a zero buffer
	// is an opaque-enough call that the write survives, and KeepAlive stops
	// the whole thing being collected out from under us.
	zeros := make([]byte, len(l.b))
	subtle.ConstantTimeCopy(1, l.b, zeros)
	runtime.KeepAlive(l.b)

	_ = unix.Munlock(l.b)
	_ = unix.Munmap(l.b[:cap(l.b)])
	l.b = nil
}

Now the asterisks. mlock keeps pages out of swap but not out of a core dump unless you also mark them, not out of a hibernation image, and not out of the hypervisor's view. Explicit zeroing overwrites the buffer you know about, and says nothing about the copies made when a value was passed by value, boxed into an interface, formatted into an error string, or moved by a stack-growth copy. And in a garbage-collected language — Python, Java, C#, JavaScript, and Go for anything that lives on the managed heap — you do not really control key lifetime at all. A Python bytes object is immutable; there is no supported way to overwrite it, and by the time you would like to, the interpreter may have made two more copies while decoding base64.

Which brings me to the argument that actually motivates this whole design. Destroying the machine is a cruder zeroization than a careful memset, and a far more reliable one. You are not asking a JVM to forget something; you are removing the address space in which the something existed, along with the kernel that mapped it. Linux hands out zeroed anonymous pages, so the guest's memory is not readable by whatever the host allocates next. No dependency's finalizer had to run correctly. Nobody had to remember to call Close on the error path. The guarantee comes from the shape of the system rather than the discipline of the code, which is the only kind of guarantee that survives a team growing.

Shared process vs container vs microVM vs KMS-only

Weakest to strongest, with the honest caveat that the container column describes general architectural properties and common defaults rather than a benchmark — a hardened, single-tenant-per-node container platform closes several of these gaps, so verify against your own platform's docs. Only the PandaStack timings are measured.

• Shared service process, all tenants — Key exposure window: the process lifetime, typically weeks, because you will cache unwrapped DEKs. Blast radius: every tenant whose key is in the cache; one heap dump, core file or disclosure bug is a fleet-wide event. Performance cost: zero, which is why it is the default. Does not solve: anything discussed in this post — it is the thing this post is about.
• Container per tenant — Key exposure window: the container's lifetime, which is honest progress if containers are genuinely per-request rather than a warm pool. Blast radius: one tenant for memory-disclosure bugs, but every tenant on that node shares one kernel, so a kernel bug or a container escape reaches the neighbours' memory. Performance cost: milliseconds on a warm image, seconds on a cold pull. Does not solve: shared-kernel exposure, and it does not stop a crash dump or a profiler from capturing the key inside the container.
• microVM per key session — Key exposure window: the duration of one operation, seconds, then the machine is gone. Blast radius: one tenant's session; a memory-disclosure bug leaks a key to code that was already inside that guest with that key. Performance cost: about 179ms p50 and 203ms p99 per session on snapshot restore, roughly 3 seconds for the one-time template cold boot, plus one KMS unwrap you were paying for anyway. Does not solve: a compromised host or a malicious operator with host access, and it introduces the snapshot and RNG-reuse traps above — you must actively refuse to snapshot these guests.
• Customer-side HSM or KMS-only — Key exposure window: none on your side; the key never enters your infrastructure. Blast radius: nothing of yours to breach, because you never had it. Performance cost: a network round trip per operation, hard payload limits of a few kilobytes, and per-call pricing. Does not solve: any workload that needs the plaintext to do real work — you cannot run full-text indexing, transcoding or analytics inside a KMS, which is the whole reason the other three rows exist.

Where a microVM is not the answer

If the operation can be performed entirely inside the customer's own KMS or HSM, do that instead, and do not let an interesting architecture talk you out of the boring correct answer. A key that never enters your infrastructure beats a key that enters it briefly, and it always will. Signing and verification, wrapping and unwrapping other keys, deriving a subkey, MACs, and encrypting small payloads — a few kilobytes of tokens, config or credentials — are all operations a modern KMS performs server-side. You send the input and receive the output, and there was never a moment when you held the key.

The microVM is for the case where you must have the plaintext or the DEK in order to do actual work, and pushing the computation into the KMS is simply not possible. Building a full-text index over an encrypted corpus. Transcoding an encrypted video. Generating thumbnails. Running analytics over a customer's encrypted event stream. Computing embeddings for retrieval. In every one of those, the payload is megabytes to gigabytes, the computation is arbitrary, and no KMS on earth will run your tokeniser. So the plaintext comes to you — and the only remaining question is what kind of machine it lands in.

There is a third case worth naming so you can dismiss it quickly: if your workload does not actually need per-tenant keys, do not build any of this. A single well-managed platform key with strong access control is simpler, cheaper and easier to operate correctly than a BYOK implementation you half-finished under sales pressure. BYOK is a real feature with real value, but it is a feature, not a security upgrade you get for free.

Audit and evidence: the underrated half of the argument

The part that sells this design internally is usually not the memory-safety story. It is that a discrete machine with a discrete lifecycle per key operation is dramatically easier to evidence than "a request in a shared pool." When a customer's security team asks what happened to their data on 3 September, the shared-pool answer is a correlation exercise across application logs, and every step is an inference. The per-session answer is a record: this sandbox was created at this timestamp on this template, its KMS token was scoped to this key with this action, the customer's own KMS log shows a Decrypt with this request ID, the exec ran for 1.8 seconds, and the machine was destroyed at this timestamp.

That chain has a useful property: the most load-bearing link is in the customer's account, not yours. Their KMS audit log is a record you cannot edit, showing every unwrap you ever requested, with the encryption context naming the tenant and object. Combine it with your own sandbox lifecycle events and you can answer scope questions with timestamps instead of adjectives.

I want to be exact about what that does and does not mean, because this is the paragraph where blog posts usually overreach. Being able to produce this evidence makes an audit conversation shorter and an incident notification more precise. It does not make you compliant with anything, it is not a certification, and it is not legal advice — I build infrastructure, and your auditor and your counsel are the people whose opinion on that question counts.

The honest limits

This defends against the failure modes above and against nothing else, so here is the list I would want a reviewer to hold me to.

• It does not defend against a compromised host. The hypervisor can read guest memory by construction — that is what a hypervisor is. If an attacker owns the host, per-guest isolation buys you nothing on that host.
• It does not defend against a malicious operator with host access. Anyone who can attach to a Firecracker process, dump its memory, or read the snapshot bucket has the key. Answer that with access control, hardware-backed audit and separation of duties, not with a VM boundary.
• A microVM is not an HSM. There is no tamper-resistant hardware, no key-never-leaves-the-boundary guarantee, no FIPS-validated cryptographic module, and no physical anti-extraction property. It is a strong software isolation boundary that happens to have a very short life. If your requirement is genuinely an HSM, buy an HSM.
• KVM and Firecracker are software and have had bugs. A microVM boundary is much narrower than a shared kernel — Firecracker's device model is deliberately tiny — but "narrower" is a probability statement, not a proof.
• Side channels are unaddressed here. Cross-VM cache and speculative-execution attacks are a real research area; mitigating them is a host and CPU concern involving core scheduling and microcode, not something a per-session guest gives you for free.
• The design fails open if you get lazy. Reuse a guest across tenants to save the 179ms, snapshot one for debugging, or let auto-hibernate freeze a live session, and you have quietly rebuilt the shared-process model with a more expensive bill.

The summary

BYOK is an envelope scheme: a per-object DEK wrapped by a KEK the customer controls in their own KMS. The strongest versions never put the KEK in your memory, and that is worth building because it gives the customer a real kill switch and a real audit log. But the unwrapped DEK and the plaintext still land on your infrastructure the moment you do useful work, and in the default architecture they land on a heap shared by every tenant, alongside a cache of everyone else's keys and an error reporter that uploads crash dumps.

A short-lived microVM per crypto session changes the blast radius of that from the fleet to one session. Snapshot restore makes it cost about 179ms, which is what turns the idea from a whiteboard diagram into something you can run per operation. In exchange you inherit two traps you must handle deliberately: never snapshot, fork or auto-hibernate a guest that has held key material, and always reseed entropy on a restored guest before it generates a key or a nonce.

And if the operation fits inside the customer's KMS, put it there and skip all of this. Destroying a machine is a crude form of zeroization, but it is far more reliable than persuading a garbage collector to forget something. Not having the key at all is more reliable still.