Answering a Security Questionnaire When You Run Customer Code

The spreadsheet arrives in week one, usually attached to an email that is otherwise very warm. It might be a CAIQ tab, a SIG Lite, or 140 bespoke rows a security team assembled from the last four reviews they ran. Somebody has to fill it in, and at a company of your size that somebody is you.

The problem is not the volume. It is that the rows were written for a product that stores records in a database and shows them to people. You do something categorically different: you take a byte sequence that a customer's model produced ninety seconds ago and execute it on your infrastructure. Half the rows are a trivial yes, a quarter are genuinely not applicable in a way that reads as evasive, and about a dozen carry the entire review. The most consequential question of the lot is not on the sheet at all, because the person who wrote the sheet has never had to think about a file that contains someone's RAM.

I answer these for PandaStack, which runs untrusted code as its whole function, so read this with that in mind. What follows is the honest version of a dozen answers, including the ones where our answer is not the flattering one. If you are on the other side of the table — the reviewer rather than the vendor — there is a companion piece linked at the bottom written from your seat.

One framing note before the questions, because it changes how you write every answer. The reviewer is not an adversary and is usually not trying to fail you. Their job is to produce a paragraph for a risk register that will survive being read aloud by their boss in six months. Founders lose these reviews in three predictable ways: over-claiming, which is fatal the moment one claim is checked; giving one-word answers on the rows that matter, which reads as not understanding your own system; and writing essays on the trivial rows, which reads as padding. The rule I have settled on is to answer the row in its own vocabulary, then add one sentence it did not ask for.

"Do you execute customer-supplied code?"

Yes. Never soften this, never reach for "we provide a managed execution environment". The reviewer already knows, it is why the questionnaire is longer than usual, and hedging on the first substantive row poisons everything after it.

The real question is the follow-up: what is the boundary. This is the one paragraph in the whole document that has to be readable by somebody who has never written kernel code, because that person is the one deciding. Here is the shape that works. In a container, every tenant's code runs against one shared Linux kernel; the boundary is the kernel's syscall surface, and the historical record of escapes is a record of bugs reached through that surface. In a hardware-virtualised VM, each tenant gets its own kernel, and an escape has to get through the hypervisor's device model instead — a far smaller and far more scrutinised surface. Our answer is the second one: every sandbox, app and managed database runs in its own Firecracker microVM with a dedicated Linux kernel on KVM, and the VMM itself is an unprivileged host process under a seccomp filter with no BIOS, no PCI and no USB emulation.

A quick test of whether your paragraph works: hand it to somebody non-technical and see whether they can ask a sensible follow-up. If it generates no follow-up you have written marketing, not an answer.

"How are tenants separated — logically or physically?"

This row is inherited from an era of racks and it forces a binary that does not describe modern infrastructure. Do not pick one. Answer in layers, and name the layer that is genuinely shared.

• Control plane — every resource belongs to an organisation, and authorisation is checked against the owning org on each request rather than inferred from possessing an identifier. API tokens are stored hashed and shown exactly once at creation.
• Network — each sandbox gets its own Linux network namespace with its own /30 and its own virtual interface. Traffic from the sandbox pool to the sandbox pool is dropped at the top of the host's forward chain, so two sandboxes cannot address each other even if they guess addresses. The whole 169.254.0.0/16 link-local range is dropped as well, which is what stops guest code reaching the cloud metadata service and helping itself to the host's service-account token.
• Compute — separate kernel, separate memory, separate virtual devices per workload; the isolation is structural rather than a configuration flag that a future deploy can quietly flip.
• Storage — each sandbox boots a copy-on-write clone of a template image, and the clone is torn down with the sandbox.

Then the sentence the row did not ask for: the physical host is shared. Two customers' microVMs land on the same machine, and if a reviewer wants single-tenant hardware, that is a commercial conversation rather than a technical impossibility. Volunteering that costs nothing and it inoculates you against the version of this discovery where they find it themselves in month three.

"Is customer data encrypted at rest?" — and the row nobody writes

The row itself is easy. Disks, snapshots, backups and stored artifacts sit on cloud-provider storage with encryption at rest. Application secrets get a second layer: sealed with AES-256-GCM before they are stored, bound as additional authenticated data to the row that owns them so a ciphertext copied into a different row simply fails to open, decrypted only in the deploy worker at the moment of injection, and redacted from build logs. If you do not yet run a KMS envelope or offer customer-managed keys, say that in the same breath rather than letting "encrypted at rest" imply a key hierarchy you do not have.

Now the row that is not on any questionnaire I have ever received, and which you should write in yourself: what about a memory snapshot?

If your platform can snapshot, fork or hibernate a running VM — and if it boots quickly, it almost certainly does — then somewhere there is a file that is a byte-for-byte image of that guest's RAM at a moment in time. Environment variables. A private key the process loaded at startup. Rows a database driver decrypted into a buffer. This is not a leak; it is how snapshot-restore works, and it is the mechanism that makes a create take milliseconds instead of seconds. It is also the most sensitive artifact you hold, and no CAIQ row goes looking for it.

Volunteer it, and answer four things: where those images live, what encrypts them, how long they persist, and the operational consequence — a secret that was resident in memory when the snapshot was taken is in the snapshot, so credentials should be fetched after restore rather than baked in before capture. Then add the tail nobody expects. When a memory image is streamed on demand from object storage rather than downloaded whole, the chunks that were faulted in are cached on the host's local disk, keyed to that specific object, and evicted on a size budget rather than by your delete call. That is a real property of a real system, and the reason to say it out loud is that a reviewer who hears it stops wondering what else you have not mentioned.

"What do you log, and for how long?"

Read the fear underneath the row. The reviewer is not asking about log retention policy. They are asking whether you keep their source code and their command output, because their engineers are about to pipe proprietary data through your exec endpoint.

So answer with the schema, not with a policy. Ours is: a metrics stream of counters (CPU, memory, network and disk bytes per sandbox), a lifecycle event stream (created, running, forked, hibernated, woken, deleted, with the template name and boot timing), and boot timings — all keyed by workspace and sandbox id, all expiring on a 90-day TTL. Separately, HTTP request metadata: method, route, status, duration, actor, IP, user agent, kept 30 days. Command text and process output are not in that pipeline; the platform records that an exec happened, not what it said.

Then the exception, because there is always one and finding it yourself is worth more than any answer you give. Application build logs are stored, because a customer whose build fails needs to read why. Every registered secret value is redacted from each line before it is written. Say that. Say too if, like us, you do not yet ship a tamper-evident audit-log export for customers — a reviewer can plan around a gap they were told about and cannot plan around one they discover.

"Can your staff access customer environments?"

The row wants the word "no". For almost every platform that offers a remote-execution API, "no" is a lie, and it is a lie that a technical reviewer can disprove in one question: how does your exec endpoint work, then?

The truthful architecture is that the host-side agent holds a key that lets it run commands inside the guests it manages. That is not a backdoor, it is the product. Anyone with root on that host has the same reach, because the guest's memory image is a file on that host's disk. Nothing in the isolation model prevents the operator; the isolation model separates tenants from each other, which is a different claim. The only technology that changes this answer is confidential computing, where guest memory is encrypted against the host, and if you do not run it you should not imply that you do.

So give a two-part answer. First, what the product exposes: no impersonation, no "view as customer" button, no support console that opens a shell in a customer workload. Our internal operations tooling reads control-plane metadata — accounts, fleet health, boot latency, flagged orgs — over a private tunnel, is not internet-exposed, and does not read customer disks. Second, the operational controls around the access that does exist: named accounts, least privilege, production access restricted to authorised personnel under confidentiality obligations, and a break-glass path that requires a reason. That answer is longer than "no" and it is the one that survives the follow-up question.

"What is your data deletion SLA?"

This is where most platforms quietly fail, and the failure is not the primary object. Everyone deletes the VM. The gap is the derived artifacts, and on an execution platform there are more of them than the reviewer imagines and, frequently, more than the vendor has enumerated.

Sit down and write the tree for a single sandbox. Ours: the copy-on-write root disk and its CoW layer, the network namespace and its firewall rules, every snapshot taken from that sandbox, every fork descended from those snapshots, the memory image tiered out to object storage while it slept, the durable volume if it backs a managed database, its backup archive, its telemetry rows, and any build logs from deploys that ran on it.

Then describe the actual mechanics, because the mechanics are what makes it credible:

1. An explicit delete kills the VM, releases the network slot, and removes the control-plane row synchronously — a follow-up GET returns 404 before the call returns.
2. Asynchronously it cascades to that sandbox's snapshots: database row, local directory and the object-storage copy; removes the VM directory and its CoW layer; deletes the memory image if the sandbox had been tiered out to object storage; and removes the durable database volume.
3. An idle auto-reap deliberately does not cascade to snapshots. A snapshot is a durable artifact the user explicitly asked for, and an idle timeout must not destroy it. Snapshots orphaned that way are expired by a sweeper after a grace period — seven days by default.
4. Deleting a managed database deletes its backup archive, base backups and write-ahead log, from object storage. This is the one people forget, and it is the one that matters most under GDPR.
5. Telemetry ages out on its own TTL rather than on delete: 90 days for metrics and lifecycle events, 30 for request metadata.
6. On termination of the agreement, remaining customer content is deleted within 30 days.

A reviewer who receives that answer knows more about your platform than one who receives "data is deleted immediately upon request", and — this is the part founders underestimate — they can tell which of the two vendors has actually looked.

"Do you hold SOC 2, ISO 27001, or a recent penetration test?"

Sooner or later a startup answers this row with something it cannot support. The two failure modes are describing yourself as "SOC 2 compliant" — which is not a status that exists, and which a reviewer will translate as "no report" — and pointing at your cloud provider's certificate as though inheritance were a thing. Your provider's attestation covers their physical and platform layers. It says nothing about you.

When the answer is no, the version that still gets you through procurement has four parts.

• The plain no, stated in public rather than only in the spreadsheet. Ours is on our security page: no audited SOC 2, no ISO 27001, no HIPAA attestation, and we do not claim certifications we have not completed. A reviewer who finds the same sentence on your website that you gave them in a document stops discounting your other answers.
• What you do maintain instead, with links: a published data processing agreement with Standard Contractual Clauses and the UK addendum, a public subprocessor list, a completed CAIQ available on request, a PCI DSS SAQ A self-assessment where all card handling sits with a payment processor, a published security.txt and a stated disclosure process.
• What they can verify without trusting you. This is the strongest card an open-source-core company holds: the agent that creates the microVMs, programs the per-sandbox networking and manages snapshots is readable under Apache-2.0. They can also test isolation themselves, inside their own organisation, within your acceptable-use rules.
• What you will commit to contractually: an audit right scoped to their own content, breach notification, advance notice of subprocessor changes with a right to object. A commitment with a lever in it beats an intention.

A "no" with four supporting artifacts is far more purchasable than a "yes" with no scope attached, and scope is where compliance answers get slippery: an attestation covering the corporate environment but not the compute platform is common and nearly useless for this particular decision. If a signed report is a hard gate for their procurement, find that out in week one and say so, rather than spending six weeks discovering it at contract stage.

One more discipline while you are here: answer these rows from the implementation, not from your own pricing page. Marketing pages describe intent and roadmap on the same axis as shipped features, and the reviewer who asks about single sign-on will ask for the metadata URL, not the plan comparison table.

Residency, subprocessors, and the rows with dates in them

"Where is customer data processed?" deserves one flat sentence. Ours: primarily in the United States, with the European Commission's Standard Contractual Clauses and the UK International Data Transfer Addendum incorporated where GDPR or UK GDPR applies to the transfer. If you cannot offer EU-only processing, say it here rather than at the DPA stage. Data residency is a common hard requirement and it is much cheaper to fail it in week one.

"What happens when you change subprocessors?" is really asking whether they get a say. Answer with the mechanism rather than the intention: a public list that is updated when it changes, and — for customers who have executed the DPA — a notification list they can subscribe to for advance notice, with the right to object on reasonable data-protection grounds and to terminate the affected service if the objection cannot be accommodated. If your answer is only "we will update the page", expect that row to come back.

"Do you have a documented incident response process?" is testing whether you have written anything down at all. The commitment that matters is notification without undue delay once you are aware, with the information you actually have at that point and updates as the investigation progresses. Resist the urge to name an hour count you have never rehearsed.

"How do you handle vulnerability disclosure?" wants four concrete things: a published contact, a stated coordinated-disclosure window (ours is 90 days), an explicit in-scope and out-of-scope list, and honesty about whether you pay. We do not run a paid bounty, we say so, and we credit reporters who want it. For a platform that runs untrusted code, the in-scope list is worth writing carefully — sandbox escape, cross-sandbox access, host agent privilege escalation, secret leakage in logs or build artifacts — because it tells a researcher which of your properties you consider load-bearing.

Build the answer library on the first deal, not the third

You will answer 140 rows for the first customer. The second arrives with a different spreadsheet asking the same twelve things in different words, and the third is a Word document with a signature block. The rows churn; the underlying truths do not. Keep them in the repository, next to the code they describe, so that a change to the code has a chance of failing a review before it ships to a customer's risk register.

# security-answers.yml — one entry per question asked more than once.

- id: executes-customer-code
  asked_as:
    - "Do you execute customer-supplied code?"
    - "Is untrusted code processed on shared infrastructure?"
  answer: >
    Yes. Every sandbox, app and managed database runs in its own Firecracker
    microVM with a dedicated Linux kernel on KVM. Tenants never share a kernel.
    The VMM runs as an unprivileged, seccomp-confined host process.
  evidence: agent/internal/sandbox/manager.go; /security#isolation
  last_verified: 2026-09-09

- id: memory-snapshot-at-rest
  volunteered: true          # no questionnaire has ever asked this row
  asked_as:
    - "Is customer data encrypted at rest?"   # the row it belongs under
  answer: >
    Yes, and note that a VM snapshot contains a byte-for-byte image of guest
    RAM. Anything resident in memory at capture time is in the snapshot, so
    fetch credentials after restore rather than before capture.
  evidence: agent/internal/snapshot/; /security#data
  last_verified: 2026-09-09

- id: deletion-derived-artifacts
  asked_as:
    - "What is your data deletion SLA?"
    - "Are backups included in deletion?"
  answer: >
    Explicit delete removes the VM row synchronously and cascades to disks,
    snapshots, tiered memory images and database backup archives. Idle reaps
    do not destroy snapshots; orphans expire after a 7-day grace period.
    Telemetry ages out on TTL (90d events, 30d request metadata). Remaining
    content is deleted within 30 days of termination.
  caveats:
    - Object-storage cleanup is best-effort; a failure leaks storage, never data integrity.
    - The host memory-chunk cache is evicted on a size budget, not on delete.
  evidence: agent/internal/sandbox/manager.go (delete path); /dpa#deletion
  last_verified: 2026-09-09

- id: attestations
  asked_as:
    - "Do you hold SOC 2 Type II / ISO 27001?"
  answer: >
    No. We hold no audited SOC 2, ISO 27001 or HIPAA attestation and state
    this publicly. We maintain a published DPA with SCCs, a public
    subprocessor list, a completed CAIQ on request, PCI DSS SAQ A, and a
    published security.txt. The substrate is Apache-2.0 and auditable.
  evidence: /security#compliance
  last_verified: 2026-09-09

The field that earns its place is the last one. An answer with a date on it and a pointer to the file that makes it true is an answer somebody can re-check after a refactor. An answer in a sales deck is a claim that ages silently until the day a customer tests it.

The pattern underneath all of it

The rows that look boring are exactly the rows where a code-execution platform differs from the software the questionnaire was designed for. Encryption at rest is a formality for a CRM and a genuinely interesting question for a platform that writes RAM to disk. Deletion is a database statement for a SaaS app and a tree of derived artifacts for anything that can fork. Staff access is a policy question for most vendors and an architectural one for anybody shipping an exec endpoint. A reviewer who has assessed forty CRUD applications and one of you will not know to ask, and the burden of that asymmetry falls on you whether or not you pick it up.

So pick it up. Volunteering those three rows costs a paragraph each and buys the only thing that matters in a vendor review, which is a reviewer who believes your other answers. It also converts the thing that makes your category frightening — that you run other people's code on purpose — into evidence that you have thought about it harder than the vendors who do it by accident.

The most credible sentence in any security questionnaire is the one that begins "we do not have". It is also the only sentence a reviewer cannot check up on you for later.