Rendering user-authored templates safely: SSTI and the microVM fix

The ticket is always reasonable. Customers want their invoice emails in their own voice, with their own logo, and support is tired of shipping a code change every time an enterprise account wants a different sign-off. So you add a textarea, wire it to the template engine you already have — Jinja2, Handlebars, Liquid, ERB, Twig, Freemarker, whichever one your stack came with — and render it against the customer's data. Two days, everyone happy.

I'm Ajay; I build PandaStack, a Firecracker microVM platform. The argument here is that most of those engines are not text formatters. They are small programming languages with a reachable object graph, and pointing one at attacker-controlled input is a code-execution decision, not a formatting decision. I'll walk the escalation from `{{ 7*7 }}` to a root shell, explain why the engines' own sandboxed modes reduce the risk without ending it, and describe the architecture I'd actually run.

You shipped a text formatter and accidentally shipped a REPL

The surfaces are all boring ones, which is exactly why they slip through review. None of them look like a code-execution feature on the roadmap:

• Customer-editable transactional email — welcome mails, receipts and dunning notices with merge fields.
• Invoice and PDF layouts — an HTML template rendered server-side and passed to a headless browser or a PDF toolchain.
• Report and export templates — a customer-defined CSV header, a branded summary, a scheduled digest.
• Slack, Teams and webhook message formats — a template that formats an alert payload before it's posted outbound.
• In-product content blocks — an admin-configurable banner, a per-tenant terms page, a checkout confirmation.

In every case a string a customer typed is evaluated by an engine running inside your application process, with your imports, your environment variables and your service credentials sitting one attribute lookup away. The engine was designed for templates written by the same team that wrote the app. You changed the author, and nothing else.

How SSTI escalates: {{ 7*7 }} to a root shell

Server-side template injection has a consistent shape. The probe is arithmetic, because arithmetic is deniable: a customer who types `{{ 7*7 }}` and sees `49` has learned the engine evaluates expressions rather than substituting variables, and has logged nothing that looks malicious. Everything after that is navigation. Here is the Python version, which is clearest because the object model is so introspectable.

from jinja2 import Template

# The feature you thought you shipped: data in, string out.
Template("Hi {{ name }}, your invoice is ready.").render(name="Ada")

# The feature you actually shipped, as discovered by a curious customer:
Template("{{ 7*7 }}").render()      # '49' - "relax, it's arithmetic"
Template("{{ config }}").render()   # inside a Flask app: the whole config object

# Step 2: an ordinary empty string is an object, and objects carry a type graph.
Template("{{ ''.__class__.__mro__[1].__subclasses__() }}").render()
# -> [<class 'type'>, ..., <class 'subprocess.Popen'>, ...] a few hundred entries

# Step 3: any function reachable from the template namespace carries __globals__,
# and __globals__ carries whatever that module imported. Such as os.
payload = "{{ cycler.__init__.__globals__.os.popen('id').read() }}"
print(Template(payload).render())
# uid=0(root) gid=0(root) groups=0(root)

Read what happened there. No memory corruption, no parser bug — the attacker used documented language features in the order the language intends. `''` is a `str`; `str` has a class; the class has a method resolution order; `object` has subclasses; one of those spawns processes. The `cycler` route is shorter still: the engine injects globals into every template, those globals are Python objects, and a Python function exposes the module namespace it was defined in. The engine did not have a vulnerability. It did its job.

The exact chain that works depends on the engine version, the globals your app registers and what your dependency tree imports, so treat any specific payload as illustrative — verify against the engine's current docs. The class of attack is what's stable: find an object, walk to a callable, walk to its module namespace, find something that touches the OS.

The same bug wearing different syntax

This is not a Python problem, and it isn't a Jinja problem. It's a property of any template language whose expressions can reach the host language's objects. The details differ; the escalation shape doesn't.

• Ruby (ERB and friends) — templates compile to Ruby, so an injected expression is Ruby, with the whole object space, `Kernel` and backticks behind it. There is no meaningful gap between "can inject" and "can run commands."
• Java (Freemarker, Velocity, and Thymeleaf-style expression languages) — the classic path reaches a class-loading or reflection primitive from an object already in scope, then instantiates something that runs a process. Reflection is the object graph here.
• PHP (Twig, Smarty) — the escalation historically ran through filters and functions that could be pointed at native callables; both have tightened this repeatedly, which tells you how often the class resurfaces.
• JavaScript (Handlebars, and anything that compiles templates to JS source) — compiling to a function makes an injection prototype- and constructor-reachable. "Get to `Function`, get to `require`" has been the pattern for years.
• Go (text/template) — a much smaller blast radius, because a template can only call what you explicitly put in the FuncMap. That is a design difference, not a hardening effort.

Why "sandboxed" engine modes help but don't finish the job

Every mature engine has an answer here, and the answers are worth using. Jinja2 ships `SandboxedEnvironment`, which intercepts attribute access and blocks the dunder attributes the escalation chains depend on. Handlebars restricted prototype access after the prototype-pollution era. Twig has a sandbox extension with allow-lists of tags, filters and methods. If you render customer templates in-process today, turning the sandbox mode on is the highest-value thing you can do this afternoon.

But look at what kind of control it is. A sandboxed environment is a denylist — or an allow-list with escape hatches — evaluated inside the same interpreter, in the same process, holding the same credentials. It works by knowing which attributes are dangerous, and that knowledge has to stay complete across every version of the engine, the runtime, and every library that might introduce a newly reachable object. Escapes in these modes have been found repeatedly, and not through sloppiness: the surface keeps moving underneath the list.

Liquid is the one that read the room. Shopify built it knowing merchants would write the templates, so it isn't a host language with guards bolted on — it's a separate language with its own object model, where a template value is a Liquid value with no defined route to a Ruby class, and filters are explicitly registered. It still needs a wall-clock timeout, because "safe from RCE" is not "safe from a pinned core". But the object-graph escalation isn't there to find.

Stop making the renderer your security boundary

The architectural move is small and it fixes the whole category: the renderer stops being a library call inside your API server and becomes a service with a boundary you control, and you make that boundary a VM. The contract narrows to one sentence — bytes in, bytes out — and "can this template reach my secrets" stops being a question about attribute denylists and becomes a question about what you put inside the guest.

Here's the honest comparison of the three postures, which is really a comparison of who has to be right:

• Security boundary — In-process, full engine: none; the template shares your interpreter, imports and process memory. Sandboxed mode: a denylist inside that same interpreter, where one missed attribute is a complete bypass. microVM render service: a hardware-virtualized guest with its own kernel, where escaping means breaking the hypervisor.
• What a bug costs — In-process: remote code execution as your app user, with every credential that process holds. Sandboxed mode: the same, one clever attribute chain later. microVM: a wasted VM and a 500 on one render.
• Loops and expansion bombs — In-process: your worker pins a core and hangs, and timeouts are cooperative at best. Sandboxed mode: identical; loop and output limits are opt-in. microVM: killed on the host's wall clock, no cooperation required.
• Secret exposure — In-process: everything in the environment and every open connection handle. Sandboxed mode: whatever the denylist forgot about. microVM: nothing, because the guest is given the data payload and literally nothing else.
• Latency cost — In-process: microseconds. Sandboxed mode: microseconds. microVM: about 179ms p50 to create from a baked snapshot, or 400-750ms for a same-host fork off a warmed baseline.
• Who has to be right, forever — In-process: nobody; it's already wrong. Sandboxed mode: the engine maintainers, about every attribute reachable through your entire dependency graph, on every upgrade. microVM: the hypervisor, about a small and heavily audited device interface.

You are trading milliseconds for a boundary that doesn't need a list to be complete. On a path that already involves generating and sending an email, or rendering a PDF, or posting to a webhook, that trade is not close.

The render service contract: data in, bytes out

Design the contract first, because the contract is what makes the isolation meaningful. A request carries the template, the engine, the data to bind and the limits — and no credentials, no connection strings, no callback URL. The response carries the rendered bytes or a structured error, never a stack trace, because a stack trace is a free map of your filesystem and dependency versions.

{
  "request": {
    "engine": "jinja2-sandboxed",
    "template": "Hi {{ customer.name }}, invoice {{ invoice.id }} is ready.",
    "data": {
      "customer": { "name": "Ada Lovelace" },
      "invoice": { "id": "INV-2291", "total": "$412.00" }
    },
    "limits": {
      "wall_clock_ms": 2000,
      "output_bytes": 1048576,
      "network": "none"
    }
  },
  "response_ok": { "ok": true, "output": "Hi Ada Lovelace, invoice INV-2291 is ready." },
  "response_err": { "ok": false, "error": "TemplateSyntaxError", "line": 3 }
}

Note what's missing from `data`: the raw customer record. Project it. If the template needs a display name and a total, the guest should never see the internal user ID or anyone else's email address. The VM stops the template from taking things; projection stops you from handing them over voluntarily. People skip the second because it's tedious.

import json
from pandastack import Sandbox

# Your harness, not the customer's. Runs inside the guest.
RENDERER = """
import json, sys, resource
resource.setrlimit(resource.RLIMIT_AS, (512 * 1024 * 1024,) * 2)

from jinja2.sandbox import SandboxedEnvironment

req = json.load(open("/workspace/request.json"))
try:
    env = SandboxedEnvironment(autoescape=True)
    out = env.from_string(req["template"]).render(**req["data"])
    if len(out.encode()) > req["limits"]["output_bytes"]:
        raise ValueError("output too large")
    json.dump({"ok": True, "output": out}, open("/workspace/result.json", "w"))
except Exception as e:
    # Class name only. Never ship the traceback back to the tenant.
    json.dump({"ok": False, "error": type(e).__name__},
              open("/workspace/result.json", "w"))
    sys.exit(1)
"""


def render(template: str, data: dict) -> dict:
    sbx = Sandbox.create(template="base", ttl_seconds=900)
    try:
        sbx.filesystem.write("/workspace/render.py", RENDERER)
        sbx.filesystem.write("/workspace/request.json", json.dumps({
            "template": template,   # hostile input, by assumption
            "data": data,           # projected fields only
            "limits": {"output_bytes": 1 << 20},
        }))
        r = sbx.exec("python3 /workspace/render.py", timeout_seconds=2)
        if r.exit_code != 0:
            return {"ok": False, "error": "render failed or timed out"}
        return json.loads(sbx.filesystem.read("/workspace/result.json"))
    finally:
        sbx.destroy()

The engine's own sandbox mode is still on in there, deliberately: the denylist is your second line, not your only one. If it holds, nothing happens. If it doesn't, the attacker gets a root shell in a disposable VM with no network, no secrets and two seconds to live.

Timeouts, loops, and expansion bombs

RCE is the headline, but availability is what will actually page you. Template languages have loops, and loops nest. Nobody needs to escape anything to write a template that renders for the rest of the year, and the billion-laughs shape — a small partial that recursively expands into a larger one — turns twenty bytes into memory exhaustion. Many are honest mistakes: a nested `for` over collections that were small in staging.

You cannot fix this inside the renderer, because the renderer is the thing that's stuck. The controls have to sit outside it, and each one has to be non-cooperative:

1. A wall-clock kill from the host. Two seconds is generous for a render, and a runaway loop dies on schedule regardless of what the guest is doing.
2. A hard memory ceiling. The guest's RAM is fixed by the template's baked snapshot, so an expansion bomb OOMs inside its own VM. Nothing on the host, and no other tenant, notices.
3. An output-size cap checked before you hand bytes back. A render that produced 400 MB of HTML has failed even though it finished; treat the cap as an error, not a truncation.
4. No network egress from the guest. A template that can reach the internet is a data-exfiltration channel and an SSRF pivot into your VPC. Deny by default and allow-list nothing unless a specific engine feature demands it.
5. A per-tenant concurrency limit on the service itself, so one pathological template can't occupy the whole pool even when each render is being killed correctly.

Making a VM per render affordable

The obvious objection is cost. A VM per render sounds absurd when the render is four milliseconds of string interpolation — and it was, right up until snapshot-restore stopped VMs from having to boot. There is no warm pool of idle machines here: every create restores a baked snapshot of an already-running machine, so a clean environment costs a restore rather than a boot.

The numbers I work with on PandaStack: a create through snapshot-restore is about 179ms p50 and 203ms p99, of which the restore step itself is around 49ms — the rest is network setup and a readiness probe. The first cold boot, before a snapshot exists for that template, is about 3 seconds, and you pay it once per template rather than per render. A same-host fork off a warmed baseline is 400-750ms; cross-host is 1.2-3.5s. Each host has 16,384 pre-allocated network slots, so the ceiling on density is memory and CPU rather than plumbing.

For batch work — the nightly invoice run, a dunning campaign, a scheduled report — the shape that pays for itself is one VM per tenant per batch. One tenant's templates are one trust domain and can share a VM; two tenants' never can. When the setup is expensive (fonts, a PDF toolchain, a warm interpreter), build it once, snapshot it, and fork per batch.

from pandastack import Sandbox

# Pay the expensive setup exactly once: interpreter warm, engine imported,
# fonts and the PDF toolchain resident.
base = Sandbox.create(template="base", ttl_seconds=900)
base.filesystem.write("/workspace/render.py", RENDERER)
base.exec("python3 -c 'import jinja2.sandbox'", timeout_seconds=60)
snap = base.snapshot()  # durable checkpoint; restorable on any host


def render_batch(tenant_id: str, requests: list[dict]) -> list[dict]:
    # One VM per tenant per batch. Same trust domain inside, never across.
    vm = base.fork()  # 400-750ms same-host, copy-on-write memory and disk
    out = []
    try:
        for req in requests:
            vm.filesystem.write("/workspace/request.json", json.dumps(req))
            r = vm.exec("python3 /workspace/render.py", timeout_seconds=2)
            out.append(
                json.loads(vm.filesystem.read("/workspace/result.json"))
                if r.exit_code == 0
                else {"ok": False, "error": "render failed or timed out"}
            )
    finally:
        vm.destroy()  # nothing survives to the next tenant
    return out

What the VM does not fix

Isolation solves execution, not output. Two things still need attention on your side of the boundary.

First, escaping. The rendered bytes are attacker-influenced by construction. If they become an HTML email, a customer can inject markup into a message your domain signs and sends — a phishing page carrying your DKIM signature. If they become an in-product content block, that's stored XSS against your own users. Turn autoescape on, sanitize against an allow-list of tags before the bytes leave the service, and treat "only that tenant ever sees it" as the assumption it is.

Second, includes. Most engines can pull in another template by name, and if the name is customer-controlled and resolution touches a filesystem, that's a path-traversal read inside the guest. In a VM with nothing sensitive on disk that's much smaller than it used to be — rather the point — but disable the loader unless you need it, and resolve includes from an in-memory map of that tenant's templates rather than a directory.

A template engine's job is to be expressive. A security boundary's job is to be boring. Don't ask one component to do both.

What I'd build on Monday

In order, cheapest first, and each step is worth doing on its own even if you never get to the next one:

1. Find every place a customer-supplied string reaches a template evaluator. Grep for the engine's from_string / compile / parse entry points and follow the argument backwards. This usually turns up two or three you'd forgotten.
2. Turn on the engine's sandboxed mode and autoescaping today. It's a small diff and it closes the trivial payloads.
3. Move rendering behind a service boundary. Even an in-process function with an explicit request/response type is progress: it forces you to name what crosses.
4. Project the data. Build the render payload from named fields, never by serializing an ORM object. This is the step that survives every future refactor.
5. Put the renderer in a microVM per render or per tenant-batch, with no egress, no secrets in the environment, a wall-clock timeout and an output cap.
6. Sanitize the output before it becomes an email, a page or a PDF, and log every render failure with the tenant and error class so you can see who is probing.

The reframe that makes it easy to justify: you are not adding a sandbox to a formatting feature. You have been running a code-execution feature since the day you shipped the textarea, and you're finally giving it the boundary one has always needed.