The Custom Code Step: Isolating User Code in a Low-Code Builder

There is a moment in the life of every visual builder when the roadmap gets honest. A customer needs to strip a currency symbol before the number goes into the CRM, or hash an email, or reshape a nested array of line items into the one JSON shape their ERP will accept — and there is no block in the palette for it. So you ship a block. Then the next one. Somewhere around block forty, someone at a whiteboard says the quiet part: we should just let them write code.

The "Run JavaScript" step is where a no-code product admits it is a code product with a very good editor. Every serious tool in the category has one, because the alternative is losing the deal over a two-line transform. It is usually the most-used block in the product within a year, and it is almost always the least-designed piece of infrastructure in the company.

I'm Ajay; I build PandaStack, a Firecracker microVM platform, so treat this as opinionated. But the argument here is narrow and specific to your product category: not "how do I run untrusted code" in the abstract, but how the custom-code step should work when a human is staring at a spinner in your editor, expecting logs to appear in a panel, and asking why they can't import a date library.

One step, three completely different contexts

The first mistake is treating the code step as one workload. It runs in three contexts with different budgets, and a design that serves only one of them feels broken in the other two.

• The test run. Someone clicked "Run step" in the editor and is watching a spinner. Total tolerance is roughly a second before it feels sluggish, logs are expected to appear as they are printed, and the code being run is, statistically, broken — that's why they're testing it.
• The production run. The workflow fires on a webhook or a schedule with nobody watching. Latency slack is generous, volume is not, and the inputs are now whatever the outside world sent rather than what the author had in mind.
• The fan-out. Someone put the code step inside a loop block over 5,000 rows from a spreadsheet. This is where every naive execution model falls over, and it is a completely ordinary thing for a user to build on a Tuesday afternoon.

The consequence: the test run is your strictest latency constraint and the fan-out is your strictest isolation constraint. Design for the test run alone and you get an in-process sandbox that a loop of 5,000 executions turns into an outage.

Why the easy versions fail

eval in the API process

The v0 of every code step. It is one line, it works in the demo, and it means arbitrary user code executes inside the process holding your database pool, your session signing key, and every workspace's connection tokens. There is no bug to find here — the design is that user code and platform secrets share an address space. The only reason it hasn't burned you is that nobody has tried.

In-process JavaScript sandboxes

The next stop is a library that promises a sandboxed JavaScript context inside your Node process. The most famous of these, `vm2`, was a genuinely heroic engineering effort: an attempt to make one JavaScript runtime distrust another JavaScript runtime living inside it, sharing its heap, its prototypes, and its garbage collector. It was maintained carefully by people who understood the problem better than most of us, and the history is a long sequence of clever escapes and clever patches — proxy tricks, prototype pollution, host objects leaking through error stacks — until its maintainers eventually stopped and pointed people toward isolate-level approaches instead. Check the repository yourself rather than taking my summary for it.

The generalizable lesson is not "that library was bad." It is that a sandbox whose boundary is enforced by the same runtime it is containing has to be right every time, against every future language feature, forever, while the attacker has to be right once. V8-isolate approaches are a real improvement — a separate heap is a genuine boundary — but you are still betting on one process, one kernel, and no memory-safety bug in a very large C++ codebase.

The shared worker pool

So you move execution to a fleet of workers. Better — the blast radius is a worker, not your API. But a pool is a shared resource, and the code step is the one block in your product that can consume it without limit. One user's `while (true)` pins a core until something reaps it; one user's attempt to parse a 900 MB export in memory triggers an OOM kill that takes down whatever else that worker was running.

There is a specific indignity here that is unique to builder products: the pool usually runs the visual steps too. So a wedged code step doesn't just break code steps — it stalls the Slack notifications and the sheet appends and the HTTP blocks belonging to customers who never enabled the code feature at all. The safest part of your product gets taken down by the most dangerous one, because they share a queue.

Containers, and the credentials nobody meant to grant

Containers are a real improvement and where most teams stop. Namespaces and cgroups give you a memory ceiling and a CPU share, which handles the noisy-neighbour half. What they don't give you is a kernel boundary: every container on that node talks to the same kernel, so one kernel bug crosses every tenant on the machine.

The part that turns a step escape from an incident into a company-ending week is rarely the kernel, though. It's the ambient environment. The container the step runs in was built by your deploy pipeline, so it inherits the platform's own worldview: a `DATABASE_URL` in the environment because that's how every other service gets one, a queue credential so it can report results, the node's cloud identity reachable at the metadata endpoint on a link-local address, and a network position inside your VPC where your internal services answer without authentication because "it's internal."

The shape that actually holds

The design I'd argue for has four properties, and the fourth is what makes the first three affordable.

1. One microVM per step execution — its own guest kernel under KVM, so an escape has to break a hypervisor rather than a language runtime, and cleanup is "delete the machine" rather than "hope the sandbox reset worked."
2. A hard wall-clock and memory budget enforced from outside the guest, where user code has no vote. A `while (true)` gets a SIGKILL, not a cooperative callback it can catch.
3. No ambient credentials. The step receives the inputs the workflow passes it and nothing else — no platform database URL, no queue token, no cloud instance identity, and specifically not the workspace's connection tokens unless the author explicitly wired one into that step.
4. Snapshot-restore, so per-execution isolation is a scheduling decision rather than a budget line item. On PandaStack a create restores a baked snapshot instead of cold-booting: p50 179ms, p99 around 203ms, with the restore step itself around 49ms. The one-time cold boot before a snapshot exists is roughly 3 seconds, paid once per template.

Point three deserves more attention than it usually gets in a builder product, because your platform is, structurally, a credential warehouse. You hold every workspace's Salesforce OAuth token, their Slack bot token, their Postgres password — that's what makes the visual blocks work. The temptation is to make the code step convenient by exposing the same connection registry to it. Don't. A code step that can enumerate connections is a code step that can exfiltrate a customer's entire integration surface with four lines of JavaScript, and it will be a customer's own employee who does it, not an attacker.

The right contract is an explicit pin: the author drags a connection onto the step, and only then does that one credential — ideally a short-lived handle rather than a raw refresh token — appear in the step's input. Everything else is invisible.

Egress: the customer's API, yes; your internal network, no

A code step that can't make an HTTP call is half a feature — the escape hatch usually exists to "call this one weird API our vendor never documented." So the answer isn't a blanket deny; it's a network position where the public internet is reachable and your infrastructure is not. Each guest gets its own network namespace, which is where the rules go.

# Applied in the guest's own network namespace, on the host side, before
# any user code runs. The step cannot edit these; it isn't in this netns.
nft add table inet stepfw
nft add chain inet stepfw out '{ type filter hook output priority 0; policy accept; }'

# 1. Cloud metadata. The single highest-value target in your account.
nft add rule inet stepfw out ip daddr 169.254.169.254 drop
nft add rule inet stepfw out ip daddr 169.254.0.0/16 drop

# 2. Private ranges: your VPC, your control plane, your Postgres, and the
#    "it's internal so it doesn't need auth" services behind them.
nft add rule inet stepfw out ip daddr 10.0.0.0/8 drop
nft add rule inet stepfw out ip daddr 172.16.0.0/12 drop
nft add rule inet stepfw out ip daddr 192.168.0.0/16 drop

# 3. Everything else is allowed, because "call our vendor's API" is the
#    feature. Log it, rate-limit it, and bill the egress.
nft add rule inet stepfw out counter

Executing one step

Here is the host-side executor. Note what crosses the boundary and what doesn't: the user's source, the previous block's output, and the connections that were explicitly pinned to this step. Nothing about your platform goes in.

import json
from pandastack import Sandbox

DEFAULT_TIMEOUT = 30


class StepFailed(Exception):
    pass


def run_code_step(step: dict, run: dict) -> dict:
    """Execute ONE custom-code step for ONE workflow run, in its own microVM.

    `step` is what the builder saved: the user's source plus the runtime
    snapshot their workspace's dependencies were baked into. `run` carries
    the upstream block's output. The workspace's connection registry, our
    database URL and our queue credentials stay out here on the host.
    """
    timeout = int(step.get("timeout_seconds", DEFAULT_TIMEOUT))

    sbx = Sandbox.create(
        template=step.get("runtime_snapshot") or "base",
        ttl_seconds=timeout + 30,  # backstop: the VM reaps itself if we crash
        metadata={
            "workspace": run["workspace_id"],
            "run": run["id"],
            "step": step["id"],
        },
    )
    try:
        sbx.filesystem.write("/run/step/user.js", step["source"].encode())
        sbx.filesystem.write(
            "/run/step/input.json",
            json.dumps(
                {
                    "input": run["outputs"][step["parent_id"]],
                    # Explicitly pinned connections ONLY. If the author did
                    # not drag it onto this step, the step cannot see it.
                    "connections": run["pinned"].get(step["id"], {}),
                }
            ).encode(),
        )

        # Wall clock is enforced from out here. `while (true) {}` gets
        # SIGKILL from a process the guest cannot reach.
        res = sbx.exec(
            "node /opt/step/harness.js /run/step/user.js",
            timeout_seconds=timeout,
        )
        if res.exit_code != 0:
            # 124 = timed out, 137 = killed (usually the memory ceiling).
            raise StepFailed(_explain(res.exit_code, res.stderr[-4000:]))

        # The result comes from a file the harness owns, never from stdout.
        # stdout belongs to the user's console.log and goes to the panel.
        return json.loads(sbx.filesystem.read("/run/step/output.json"))
    finally:
        sbx.destroy()  # secrets, temp files and whatever they left behind

The harness is your code, not theirs. It defines the calling convention, turns a thrown exception into a structured failure your UI can render, and writes the result to a path the user's code doesn't choose. Keeping the result on a file channel rather than scraping stdout matters more here than in a generic function runner, because in a builder the user's `console.log` output is itself a product feature.

Getting logs back into the panel while it runs

This is what separates a code step people enjoy from one they tolerate. When a user hits Run, their `console.log` lines need to appear in the panel as they happen, not in a single dump forty seconds later — because half the time the step never finishes, and the logs are the only debugging tool they have.

So the executor above needs a streaming sibling: exec over an event stream rather than a blocking call, relayed to the browser. Three things bite people here.

// Server-side relay: sandbox exec stream -> the editor's output panel.
// Runs on your API, so it also gets to be the place you enforce caps.
const MAX_LOG_BYTES = 256 * 1024;
const MAX_LINE = 8 * 1024;

export async function relayStepLogs(
  sandboxId: string,
  runId: string,
  send: (event: PanelEvent) => void,
) {
  const res = await fetch(
    `${API}/v1/sandboxes/${sandboxId}/exec/stream`,
    {
      method: "POST",
      headers: {
        Authorization: `Bearer ${process.env.PANDASTACK_API_KEY}`,
        "content-type": "application/json",
      },
      body: JSON.stringify({
        cmd: "node /opt/step/harness.js /run/step/user.js",
      }),
    },
  );

  let budget = MAX_LOG_BYTES;
  let seq = 0;

  for await (const ev of sse(res.body!)) {
    if (ev.event === "exit") {
      send({ type: "exit", runId, code: Number(ev.data) });
      return;
    }

    // 1. Attacker-controlled text. It is going into your DOM, so it is
    //    a string payload with a stream tag -- never innerHTML, and
    //    strip ANSI/control bytes so nobody paints fake UI in the panel.
    const text = sanitize(ev.data).slice(0, MAX_LINE);

    // 2. A log-flood is a denial of service against your own frontend.
    //    Cap the run, then say so instead of silently truncating.
    budget -= Buffer.byteLength(text);
    if (budget <= 0) {
      send({ type: "truncated", runId });
      return;
    }

    // 3. stdout and stderr arrive on separate channels but the user
    //    thinks in one timeline. Tag the stream, keep one sequence.
    send({
      type: "log",
      runId,
      seq: seq++,
      stream: ev.event === "stderr" ? "stderr" : "stdout",
      text,
    });
  }
}

The sanitization line is not paranoia theatre. The output panel is a place where one user's arbitrary string is rendered inside your application's UI — that is the textbook shape of a stored XSS, and in a shared workspace the string author and the string viewer are frequently different people.

"Can I use dayjs?"

This arrives about a week after the feature ships, and how you answer it determines your latency story forever. Three options, one of them good.

Installing on demand inside each execution is the obvious approach and the worst one: you've added a package-manager round trip to a user-facing step, made your egress policy include a package registry, and handed every workspace a supply-chain install. A curated set of preinstalled libraries is a reasonable v1 — fast, auditable, and guaranteed to generate a support queue asking for library forty-one.

The version that scales is a per-workspace runtime: the workspace declares a dependency manifest, you build it exactly once in a sandbox, snapshot the machine after the install, and every subsequent execution restores or forks from that snapshot with the packages already on disk and, in the fork case, already in page cache.

import json
from pandastack import Sandbox


def bake_workspace_runtime(manifest: dict) -> str:
    """Install a workspace's declared libraries ONCE. Returns a snapshot id
    that later step executions boot from directly."""
    builder = Sandbox.create(template="base", ttl_seconds=900)
    try:
        builder.filesystem.write(
            "/run/step/package.json", json.dumps(manifest).encode()
        )
        # --ignore-scripts: a postinstall hook is code execution too, and
        # this one runs at build time on a machine we keep around.
        res = builder.exec(
            "cd /run/step && npm install --ignore-scripts --no-audit",
            timeout_seconds=600,
        )
        if res.exit_code != 0:
            raise BuildFailed(res.stderr[-8000:])
        return builder.snapshot().id
    finally:
        builder.destroy()


def run_fanout(snapshot_id: str, rows: list, concurrency: int = 32):
    """A loop block over 5,000 rows: fork a warm parent per item so every
    row still gets its own kernel, without re-paying setup 5,000 times."""
    parent = Sandbox.create(template=snapshot_id, ttl_seconds=900)
    try:
        for batch in chunks(rows, concurrency):
            children = [parent.fork() for _ in batch]
            try:
                yield from execute_batch(children, batch)
            finally:
                for child in children:
                    child.destroy()
    finally:
        parent.destroy()

Be honest about the trade in that second function. A same-host fork is 400–750ms, slower than a straight snapshot-restore create — forking is a state optimization, not a latency one. Reach for it when the parent holds something you don't want to rebuild per item: a warmed interpreter, a large dataset in memory, an authenticated client. If your items need nothing but the packages on disk, plain creates from the workspace snapshot are simpler and faster. (Cross-host forks run 1.2–3.5s, so keep a fan-out on one host.)

Either way, the fan-out needs a per-workspace concurrency cap, because a loop over 5,000 rows is a load test that one of your users will run without telling you. Capacity per host is bounded by memory and CPU rather than by networking — a PandaStack agent pre-allocates 16,384 network slots — so the cap is a number you choose, not a limit you discover.

The four options, side by side

Softest boundary to hardest. If you're evaluating specific libraries or runtimes, verify their current isolation properties against their own documentation.

• Isolation boundary — In-process eval: none; user code shares your heap. In-process JS sandbox: enforced by the runtime that's being sandboxed, so it must be right every time forever. Container per execution: namespaces plus cgroups on a shared host kernel. microVM per execution: a separate guest kernel under KVM hardware virtualization.
• Blast radius of an escape — In-process eval: your API process, its database pool and every workspace's tokens. In-process JS sandbox: the same process, one clever prototype trick later. Container per execution: the node, plus whatever its cloud identity and VPC position reach. microVM per execution: one throwaway guest holding one step's inputs.
• Runaway loop or memory hog — In-process eval: takes the API down with it. In-process JS sandbox: a tight loop can starve the event loop that was supposed to time it out. Container per execution: cgroup limits hold, though page cache and IO are shared. microVM per execution: a fixed vCPU and RAM boundary, killed from outside on wall clock.
• Test-run latency — In-process eval: microseconds. In-process JS sandbox: milliseconds to build a context. Container per execution: hundreds of milliseconds warm, seconds on a cold image pull. microVM per execution: p50 179ms via snapshot-restore, p99 around 203ms.
• Per-workspace libraries — In-process eval: one dependency set for everyone, forever. In-process JS sandbox: same, plus whatever the sandbox refuses to let through. Container per execution: an image per workspace and a build pipeline to match. microVM per execution: install once, snapshot, restore or fork per execution.
• Operational cost — In-process eval: none, until the incident review. In-process JS sandbox: low, plus an open-ended obligation to track escape research. Container per execution: an orchestrator, a registry, and image lifecycle. microVM per execution: many short VM lifecycles to schedule and clean up — the real cost, and the reason to buy the substrate rather than build it.

Every isolation model works until a customer writes the code that breaks it. The question is only whether the answer that day is "one VM died" or "we are rotating every workspace's OAuth tokens this weekend."

When you don't need any of this

If what your users are writing isn't really code, don't build a code-execution platform. A field mapping, a filter predicate, a template string with two interpolations — evaluate those with a purpose-built expression language that has no I/O, no imports and no loops. Something non-Turing-complete is genuinely safe in-process, because it cannot do anything. Plenty of builder products serve most of the "I need a transform" demand with an expression field and never ship a runtime.

The line is sharp: the moment a user can `import`, open a socket, spawn a process, or write an unbounded loop, you are hosting untrusted code and the boundary has to be an OS-or-hardware boundary. A microVM is the strongest of those that still starts fast enough to sit inside an interactive test run — which is the only reason this design is available to you at all. Ten years ago the honest answer was "put it in a queue," and everyone shipped `eval` instead.

If you're going to ship the step — and you are, because your competitor already did — ship it with the boundary drawn where a mistake costs you one guest. The feature is too popular to stay dangerous.