SOC 2 for a Platform That Runs Other People's Code

Two things tend to arrive at roughly the same moment in a company's life. A customer's security team sends over a questionnaire with a column headed "Evidence," and someone internally says the words "we should probably start SOC 2." If your product runs code your customers wrote — an AI agent sandbox, a CI runner, a hosted notebook, a PaaS that builds whatever is in the repo — the next few months will be more interesting than they would be for a CRUD app, and not in the fun way.

I'm Ajay; I build PandaStack, a Firecracker microVM platform, and I talk to a lot of teams whose core feature is executing a stranger's program. This post is what an auditor actually asks for in that situation, and how to already have the answer sitting in a database rather than reconstructing it under deadline. The premise is uncomfortable enough to say in the first paragraph: the audit is not about whether you are secure.

What the auditor is actually testing

An auditor is not evaluating your architecture on its merits. They are running a two-step check, and internalizing it early saves months. Step one: you described a control, in your own words, in your own system description. Step two: the auditor tests whether that control, as you described it, operated consistently across the observation window. Not whether it is a good control. Whether it did the thing you said it does, every time, for the whole window.

That is the gap the entire engagement lives in. Engineers turn up with a threat model, a diagram, and a well-founded belief that their isolation is strong. The auditor wants the record for every one of the four thousand-odd times the control fired, and a way to sample it at random. "We are careful" is unauditable. "Every execution runs in a fresh microVM that is destroyed afterwards" is testable — and if you cannot produce a record of each execution and the machine it ran on, then as far as the report is concerned the control does not exist. It exists as an undocumented feature of the founder's memory.

The shape of the thing, briefly and correctly

SOC 2 is built on the Trust Services Criteria. Security — often called the common criteria — is mandatory; every report includes it. Availability, Confidentiality, Processing Integrity and Privacy are optional categories you opt into. Most companies start with Security alone, add Availability when a customer contract mentions uptime, and add Confidentiality when they handle material the customer considers theirs. Privacy is a much bigger commitment than teams expect. Every category you add is more surface to evidence, so claim none you do not need.

Type I versus Type II is a question of tense. A Type I says the controls were suitably designed at a point in time — a photograph. A Type II says they operated effectively across an observation window, commonly three months for a first report and up to twelve thereafter. Type I is a defensible stepping stone, and also the one enterprise procurement has learned to be unimpressed by, because design without operation is a promise rather than a result.

The part that surprises people most is that you write the control descriptions. No standards body hands down a checklist: you produce a system description and a set of control statements, and the auditor tests what you wrote. Write one loosely — "access to production is restricted to authorized personnel" — and you have signed up to define authorized, produce the roster, and show the restriction held for every access in the window. A vague control is not a hedge. It is a trap you set for yourself and walk into ten months later.

The controls that are hard because you run other people's code

Every company doing SOC 2 deals with laptops, offboarding and vendor reviews. Those are tedious and thoroughly trodden. These are the ones that get materially harder the moment your product's job is to execute a program someone else wrote.

Logical access and tenant isolation

This is the control your customers actually care about and the one you will spend the most time describing. The auditor's question, stripped of ceremony, is: what stops one customer's execution from reaching another customer's data, and how do you know it held? You need an answer in one sentence that a non-specialist can read and a penetration tester can attack.

"Each execution runs in a dedicated Firecracker microVM with its own guest kernel and network namespace, holds no credentials for any other tenant, and is destroyed when the execution ends" is such a sentence: every noun in it either exists or does not. Compare "workloads are isolated using containers with a hardened seccomp profile and network policies." Both may be perfectly defensible in reality, but only the first ends the conversation. The second starts a long one about which policies, enforced by which layer, in which CNI mode, and what to make of the log agent reading every workload's stdout. A boundary you can name in a sentence is cheaper to evidence than one that emerges from fourteen configuration objects, any of which an engineer can change on a Thursday without knowing it was a compliance boundary.

Audit logging: who ran what, when, for whom, with what result

The Security criteria expect you to log security-relevant events and be able to review them. For a code-execution platform the security-relevant event is the execution itself, and the fields are not mysterious: which tenant, which human or API key initiated it, what ran, on which image, inside which boundary, how it ended, and when. Emit that to a store that outlives the machine, with a written retention period.

The shape matters more than the exact schema. Give the auditor something they can filter, count and sample — one JSON object per line, one line per lifecycle transition.

{
  "event": "sandbox.created",
  "event_id": "01J9K2QW7YB3M4N5P6R7S8T9V0",
  "occurred_at": "2026-09-05T14:22:07.412Z",
  "tenant": { "org_id": "org_8f21c4", "org_slug": "acme-labs" },
  "actor": {
    "type": "api_key",
    "id": "key_1f0a9c",
    "subject": "svc-notebook-runner",
    "source_ip": "203.0.113.44"
  },
  "action": "sandbox.create",
  "resource": {
    "type": "sandbox",
    "id": "sbx_2f7a1d9e",
    "request_id": "req_c41b7e20"
  },
  "isolation": {
    "boundary": "firecracker-microvm",
    "guest_kernel": "5.10",
    "network": "dedicated-netns",
    "egress_policy": "deny-all",
    "shared_with_other_tenants": false
  },
  "image": {
    "template": "code-interpreter",
    "generation": "2026-08-29T09:14:00Z-a71c3f"
  },
  "outcome": { "status": "started" },
  "retention": { "class": "security-audit", "expires_at": "2027-09-05T14:22:07Z" }
}
{
  "event": "sandbox.destroyed",
  "event_id": "01J9K2R4XA0C1D2E3F4G5H6J7K",
  "occurred_at": "2026-09-05T14:22:15.824Z",
  "tenant": { "org_id": "org_8f21c4", "org_slug": "acme-labs" },
  "actor": { "type": "api_key", "id": "key_1f0a9c" },
  "action": "sandbox.destroy",
  "resource": { "type": "sandbox", "id": "sbx_2f7a1d9e" },
  "outcome": { "status": "completed", "exit_code": 0, "reason": "client_kill" },
  "lifetime_ms": 8412,
  "data_disposition": "guest_memory_and_disk_reclaimed",
  "retention": { "class": "security-audit", "expires_at": "2027-09-05T14:22:15Z" }
}

The failure mode here is specific and extremely common: the only record of an execution is a log file inside the sandbox, or on the host that ran it. Both are things you deliberately destroy — the destruction is the other control you are proud of. You cannot evidence a control using data you deleted as part of that control. Events go out over the wire, at creation and at destruction, to a store append-only enough that you can say in one sentence who is able to alter it.

Two details auditors reliably probe. Retention: name a period, enforce it, and be able to produce a record from the first day of the window, not only the last thirty days. And clock integrity, a genuine trap in snapshot-restore architectures — a guest restored from an image baked in April believes it is April, so every timestamp it writes is fiction until something syncs it. Timestamp on the control plane, or sync on restore and say so.

Change management, which is more mechanical than you fear

The control is that production changes are reviewed and traceable, and the evidence is mercifully mechanical: branch protection settings requiring review and passing checks, the pull request history, and CI run records. Export that configuration when the window opens and again when it closes — those settings are not documentation about the control, they are the control. Two things break it: emergency direct pushes to main, which are fine if you have a documented emergency path and write the ticket afterwards, and deploys that correspond to no commit at all — a flag flipped by hand, an environment variable edited in a console. Those are production changes too. The auditor does not want your reasoning about why it was safe. The auditor wants the ticket.

Vulnerability management, and the image that never ages out

Here the architecture that makes your isolation story easy makes another control quietly harder. Your product's real operating system is the template image, baked once and restored on every execution. Nothing in it gets patched by a nightly upgrade, because nothing runs long enough to run one. A base image baked six months ago and restored a million times has an immaculate uptime record and a six-month-old libc. That is vulnerability management at scale, in the wrong direction.

So the control has to be about the bake, not the host. Describe a rebake cadence — a period plus a trigger for critical advisories — scan the image during the build, keep the build records, and version the images so an execution event can name the generation that ran. That last part is what makes it evidenceable: when the auditor asks whether a fix was actually in production, you answer with a query over which generations appear in events after the rebake date, not with a memory of having rebuilt everything.

Encryption in transit and at rest — including the snapshots

In transit is the easy half: TLS at the edge, an authenticated channel between services, a policy statement. At rest is where this shape adds surface most teams have never inventoried. A snapshot is a copy of guest memory plus device state plus a disk, so if your platform snapshots running machines for fast restore, hibernation or forking, those files hold whatever was in RAM at capture time — the customer's environment variables, their API tokens, whatever their program had just decrypted. The same goes for durable volumes. Inventory both stores alongside your primary database: encryption, access control, an audit trail, a retention schedule. Teams miss this because snapshots feel like infrastructure rather than customer data. They are customer data, in a format unusually convenient to copy into a debugging bucket.

Data deletion and retention

"The machine is destroyed and its disk reclaimed at the end of every execution" is one of the strongest sentences you can put in a system description, because it converts a promise about behaviour into a property of the lifecycle. It is only worth having if you can evidence it: a destruction event per sandbox, a TTL the platform enforces rather than a cleanup routine your orchestrator remembers to call, and an alert on machines that outlive their TTL with no matching destruction record. A cleanup function you believe always runs is a belief, and beliefs are not sampled.

The rest, quickly

The remaining common criteria are not code-execution specific, and should not ambush you in month eleven. Incident response: a written plan, a severity scale, and real incidents with timelines — a window with zero incidents and zero drills raises the question of whether anyone would notice one. Backups: backing up is not the control, restoring is, so schedule a restore test and keep the result. Vendor management: a subprocessor list with their reports reviewed and dated. Personnel: background checks where lawful, training with completion records, onboarding and offboarding checklists that produce tickets. Plus a dated risk assessment. None of it is hard, and all of it is easy to have quietly not done.

Control, question, artifact

Consolidated, the mapping looks like this in practice — the control you write, the thing the auditor asks for, and the artifact you actually hand over.

• Tenant isolation — asked: describe the boundary and show it held for every execution. Handed: the one-sentence boundary description, a per-execution record naming machine, tenant and image, the egress policy configuration, and a penetration test aimed at the boundary itself.
• Execution audit logging — asked: show me every execution in the window and who initiated it. Handed: a queryable event store with tenant, actor, action, resource, image generation, outcome and timestamp, plus a written retention period and a record from day one.
• Change management — asked: prove this deployed change was reviewed. Handed: exported branch protection settings, the pull request with its approval, the CI run, the artifact-to-commit mapping, and a ticket for every emergency change.
• Vulnerability management — asked: how do you know production is not running a six-month-old image. Handed: a rebake cadence, scan output per build, and a query showing which template generations appear in events after the rebake date.
• Encryption at rest — asked: list every place customer data comes to rest. Handed: an inventory that includes the snapshot store and durable volumes, not only the primary database, with encryption and access controls for each.
• Data deletion — asked: prove the data is gone when you say it is. Handed: a destruction event per sandbox, a platform-enforced TTL, and a reconciliation report showing no machine survived its TTL across the window.
• Access to production — asked: who can reach it, and did anyone else. Handed: an identity provider group export, MFA enforcement settings, access logs for the window, and offboarding tickets whose dates line up with the departures.

Why per-execution isolation makes the audit easier, honestly framed

There is a real advantage here, and it is worth naming precisely rather than overselling. Two things get genuinely cheaper, and a third — the conversation itself — gets shorter, because most of the cost in an isolation discussion is the number of sentences required before an auditor can form an opinion at all.

Fewer long-lived systems in scope. A fleet of persistent per-customer workers is a fleet of machines that each need hardening evidence, patch records, access control and monitoring. A population of machines that exist for one execution has, at any moment, almost nothing on it and nothing that persists across tenants. What you describe and evidence becomes the control plane and the hosts, not the workload population — and the workload population is the part that grows with revenue.

A shorter retention story. When customer data exists in a machine for the length of a job and then stops existing along with it, questions about residual data and cleanup collapse into questions about lifecycle, which you answer with events rather than assurances. This is only a viable compliance posture if it is also a viable latency posture: on PandaStack every create restores a pre-baked snapshot rather than booting, at roughly 179ms p50 and 203ms p99, with the three-second cold boot paid once at bake time.

Instrument for the auditor on day one

The highest-leverage thing you can do, and it costs almost nothing before the window opens: emit a structured lifecycle event for every execution, carrying the fields an auditor will ask you to filter on. Tenant, actor, timestamp, outcome, boundary, image. If you already tag sandboxes with metadata for operational reasons you are most of the way there — the metadata is the join key between your product's world and your evidence store.

import os

from pandastack import Sandbox

# One execution, one machine, two audit events. The sandbox is the unit of
# isolation AND the unit of evidence -- keeping those the same thing is what
# makes an auditor's sample answerable in one query instead of three.

TEMPLATE = "code-interpreter"
TEMPLATE_GENERATION = os.environ["TEMPLATE_GENERATION"]  # pins the image


def run_customer_code(org, actor, source_ip, request_id, source):
    sbx = Sandbox.create(
        template=TEMPLATE,
        # A retention control the PLATFORM enforces. If this process dies
        # mid-run, the machine still goes away on schedule. A TTL is
        # evidence; a finally block is a hope with good intentions.
        ttl_seconds=300,
        metadata={
            # NON-SENSITIVE IDENTIFIERS ONLY. Metadata comes back from list
            # APIs and lands in operational logs that live outside whatever
            # boundary you drew -- so a tenant slug and a request id belong
            # here, and customer content never does.
            "org": org.slug,
            "request": request_id,
            "purpose": "notebook-exec",
        },
    )

    audit.emit(
        event="sandbox.created",
        tenant={"org_id": org.id, "org_slug": org.slug},
        actor={"type": actor.kind, "id": actor.id, "source_ip": source_ip},
        resource={"type": "sandbox", "id": sbx.id, "request_id": request_id},
        isolation={"boundary": "firecracker-microvm", "egress_policy": "deny-all"},
        image={"template": TEMPLATE, "generation": TEMPLATE_GENERATION},
        outcome={"status": "started"},
    )

    try:
        sbx.filesystem.write("/work/main.py", source)
        run = sbx.exec("python /work/main.py", timeout_seconds=120)
        return {"stdout": run.stdout, "stderr": run.stderr, "exit": run.exit_code}
    finally:
        # Destroyed, not reset. Guest memory and disk stop existing together,
        # which is the sentence in the system description -- and the event
        # below is the only proof of it that survives the machine.
        sbx.kill()
        audit.emit(
            event="sandbox.destroyed",
            tenant={"org_id": org.id, "org_slug": org.slug},
            resource={"type": "sandbox", "id": sbx.id},
            outcome={"status": "completed", "reason": "client_kill"},
            data_disposition="guest_memory_and_disk_reclaimed",
        )

Two things in that snippet are the compliance content rather than the code. The metadata carries non-sensitive identifiers only, because metadata comes back from list APIs and lands in operational logs outside whatever boundary you described. And the TTL is set at creation, so the machine dies on schedule even when the process that created it does not survive to clean up — which matters because the unhappy path is where cleanup routines do not run, and the unhappy path is what a sample eventually lands on.

Expect sampling, and expect it to be arbitrary

An auditor will not read four thousand events. They will pull a sample — for a population like that, something in the region of twenty-five items, though the approach and the size are the auditor's call — and ask you to substantiate every one. That changes what "good evidence" means. It is not a dashboard showing a healthy aggregate. It is the ability to take an arbitrary identifier from anywhere in your window and reconstruct that single execution: who initiated it, when, on which image generation, inside which boundary, how it ended, and when the machine was destroyed.

Which leads to the tactical advice. Do not build evidence out of screenshots — a screenshot proves the state of a setting on the day someone took it, and an auditor testing a window needs the window. Prefer exports and queries you can re-run. Keep tickets for the human-shaped controls, because a ticket is a dated artifact with an actor and an outcome, which is exactly the shape of evidence. And start emitting events before the window opens: you cannot retroactively instrument a period that has already elapsed, and the window is the one input you cannot buy your way out of.

What a compliance automation vendor does, and does not, do

Vanta, Drata and their peers are useful, and most teams doing this should probably use one. Be clear about what you are buying. They collect and monitor evidence, map it to criteria, chase policy acknowledgements, and tell you which laptop is missing disk encryption. They are a very good filing system with alarms attached. They do not design your controls, they cannot describe your isolation boundary for you, and no integration they ship knows anything about your execution events unless you send them. The hard parts of this post exist only because your product runs other people's code, which is exactly why no integration covers them. Bring your auditor in early and ask what they will want to sample.

The summary

Reduce it to a sequence. Decide which criteria you are claiming, and write control descriptions that are narrow and testable rather than broad and comforting, because you will be tested on your own words. Identify the handful that are hard specifically because you execute customer code — isolation, execution logging, image freshness, snapshot encryption, deletion — and design the evidence for each before the window opens. Emit a structured event for every execution to a store that outlives the machine. Keep tickets for everything a human does. Expect to be sampled, and make sure a single identifier can be traced end to end by someone who has never seen your codebase. The architecture genuinely helps. But the report is not issued for the architecture. It is issued for the records.

The auditor does not want your threat model. The auditor wants the ticket, the log line, and the twenty-four other samples that look exactly like it.