Modal vs RunPod: Serverless Python or Raw GPU?

"Modal vs RunPod" gets searched at a very specific hour of a very specific week: the week your GPU bill stopped being a rounding error and became a lifestyle choice. Both names come up in the same threads, both promise GPUs without a datacenter, and from a distance they look like two doors into the same room. They aren't. Modal sells you a programming model that happens to have GPUs behind it. RunPod sells you GPUs that happen to have a programming model in front of them. Which of those two sentences describes your team is the decision, and most of the feature-by-feature arguing that follows is downstream of it.

I'm Ajay. I built PandaStack, which is not a GPU cloud and is not competing with either of these for your training budget — I'll say more about that near the end, and you're welcome to skip it. That disclosure is also why I can be reasonably even-handed here: I don't need you to pick one. This post is about matching workload shape to platform shape, not about crowning a winner.

What each one is actually for

Modal: your Python function, deployed, with a GPU attached to it

Modal's centre of gravity is serverless Python. You decorate a function, describe the container image in the same Python file that contains the function, and the platform takes it from there — build the image, ship the code, run it once or fan it out wide. GPUs are a property you request on the function rather than a machine you rent, reason about, and remember to turn off. Around that sit the things a batch-and-inference team needs anyway: autoscaling, scheduled jobs, web endpoints, queues, volumes for the state that has to outlive an invocation.

The direction of the abstraction is the point. Modal wants your code to come to it, and in exchange infrastructure disappears behind the decorator. Image-as-code is a genuine ergonomic win — no separate Dockerfile drifting quietly out of sync with your requirements file, no cluster to babysit, no bastion host that only one person knows the SSH config for. The cost is that you are buying into a model. Your unit of work becomes an invocation: it starts, it runs, it returns, it goes away, and anything durable is something you declared on purpose.

RunPod: GPU capacity you rent and drive

RunPod's centre of gravity is GPU infrastructure. The headline primitive is a pod — a GPU container you start, connect to, and use, closer in spirit to renting a machine than to deploying a function. You bring your own image, you get a shell, and what happens inside is your business. RunPod layers a serverless-endpoint product on top for request-driven inference with autoscaling, and it distinguishes between capacity tiers — a community-supplied pool versus a more controlled secure-cloud tier — which is a supply-side distinction Modal doesn't really expose to you because Modal isn't asking you to think about where the silicon came from.

That framing has real consequences in both directions. You keep control: any framework, any CUDA version, any weird pinned dependency your research code needs, a long-lived box you can SSH into at 2am to see what actually died. You also keep responsibility. Nothing turns itself off out of politeness, and the classic RunPod war story is the H100 someone started on a Friday afternoon that stayed warm all weekend, faithfully computing nothing at all. That's not a platform defect — it's the natural consequence of renting a machine rather than deploying a function, and it's a trade some teams make happily and others discover the hard way.

The programming model is the real fork in the road

Here is roughly the shape of Modal's API — illustrative only, not copied from their docs, and their surface changes, so check the current docs before writing anything real against it. The thing to notice isn't the syntax, it's that there is no machine anywhere in it.

# Illustrative of the SHAPE of a decorator-driven serverless-Python API.
# Not a working snippet — check Modal's current docs for real syntax.
import modal

app = modal.App("summarizer")

image = modal.Image.debian_slim().pip_install("transformers", "torch")

@app.function(image=image, gpu="A10G", timeout=600)
def summarize(doc: str) -> str:
    from transformers import pipeline
    summarizer = pipeline("summarization")
    return summarizer(doc, max_length=120)[0]["summary_text"]

@app.local_entrypoint()
def main():
    # Fan out over many inputs; the platform decides how much hardware appears.
    for out in summarize.map(load_documents()):
        print(out)

There is no provisioning step, no instance ID, no teardown. The GPU is an argument. If that reads as obviously correct to you, you probably want Modal, and the rest of this post is a formality. If it reads as a cage — because your training loop needs a specific driver stack, because you want to watch nvidia-smi while it runs, because you'd rather own the box for six hours than hand the scheduler your process — you probably want RunPod, and that's an equally legitimate instinct rather than a failure to appreciate abstraction.

The axes that actually decide it

Qualitative on purpose. The shapes below are stable; the specifics under them are not, and I'd rather be right in six months than precise today.

• Programming model — Modal: decorate a Python function, the platform deploys and scales it; the image is declared in code. RunPod: bring a container and run it on a pod you control, or wrap a handler behind a serverless endpoint; the image is yours to build and manage.
• GPU access — Modal: a property you request on a function, abstracted away from any particular machine. RunPod: closer to renting the accelerator itself, with an explicit distinction between community-supplied and more controlled secure capacity. On both, availability and price move week to week — verify on their current pricing pages, not here.
• How much infra you manage — Modal: very little by design; that's the product. RunPod: meaningfully more — images, drivers, process supervision, storage, and the discipline to stop things you started.
• Autoscaling — Modal: core to the model, including scale-to-zero between invocations and wide fan-out over inputs. RunPod: available through its serverless endpoints; a long-running pod is a machine you own and does not scale itself down out of kindness.
• State and storage — Modal: ephemeral per invocation, with volumes and object storage as the explicit durable layer. RunPod: pods carry persistent and network volume options, and a long-lived pod keeps a filesystem the way a server does.
• Untrusted-code isolation — Both document their own execution and security models, and neither should be assumed from a blog post, including this one. If you plan to run model-generated or customer-supplied code, make each vendor tell you exactly where the boundary sits — container, VM, or hardware virtualization — in writing, and verify against their current security documentation.
• Best fit — Modal: Python-first teams shipping batch jobs, scheduled pipelines, and inference endpoints who want infrastructure to vanish. RunPod: teams optimising for GPU cost and control, running long training jobs, or needing an environment too specific to fit a managed function model.

Cold starts: everyone is timing a different event

Cold start is the most misread number in this whole category, because the platforms aren't measuring the same thing. On a serverless-function platform, cold start means "how long until my code's first line runs," and a warm container makes it nearly free. On a rented pod, cold start is mostly a question you asked once at the beginning and then stopped asking, because the machine simply stays up — you've converted a latency problem into a utilisation problem. On a serverless GPU endpoint, cold start includes the part nobody's marketing page enjoys: getting multiple gigabytes of model weights onto the accelerator before the first token appears.

So the useful question isn't whose cold start is lowest. It's how many times your loop pays it, and what's actually in it. A three-second start paid once per batch job is invisible. The same three seconds paid per user-facing inference request is your entire product. And weight-loading time is usually the dominant term for large models regardless of platform, which means your image layout, your weight-caching strategy, and where those weights physically live will move your p99 more than the vendor choice will. Benchmark with your model and your real request pattern before believing anyone's number, mine included.

Price, supply, and the honest bit about both

The genuine structural difference is what you're paying for. A serverless model bills you for compute time actually consumed, which is a very good deal for bursty and irregular workloads and a progressively worse one as your utilisation approaches full. Renting capacity bills you for the machine, which is a very good deal at high sustained utilisation and a catastrophic one at low utilisation — because an idle rented GPU costs exactly the same as a busy one, and it will not mention this to you.

That crossover point, not the headline rate, is the number worth computing. Estimate your GPU-hours per month and your realistic duty cycle, then price both shapes against current published rates. Teams reliably get this wrong in one direction: they compare hourly stickers, pick the cheaper one, and then discover their duty cycle was 12% and they've been renting a very expensive space heater. Also account for the operational tax on the control-heavy option — someone's time spent on images, drivers, and stopping forgotten pods is a real line item even though it never appears on an invoice.

Neither of these is what you want if…

There's a workload that shows up in these searches by accident, because "AI infrastructure" is one phrase covering several unrelated problems. If any of the following is your actual situation, both of these platforms are the wrong shape and you should stop comparing them:

• You're running code an LLM just wrote, on CPU, and the threat model is "assume it's hostile." That's a sandboxing problem with an isolation-boundary requirement, not a GPU-capacity problem. Ask for a hardware boundary per execution, not a bigger accelerator.
• Your creates are on the hot path of an agent loop — dozens of short executions per task, each needing a clean environment. You're optimising for create latency and per-execution cost, which is a different design target than either GPU model.
• You need a long-lived stateful app with a database attached — a web service, a background worker, a Postgres instance that has to still exist tomorrow. That's application hosting, and a function platform or a GPU rental is an awkward substrate for it.
• You need to branch a running environment — take one warmed machine and explore several candidate outcomes from the same state. Neither a per-invocation function nor a rented pod exposes that operation.
• The workload is plainly CPU-bound and someone put a GPU in the requirements doc because it's 2026. Check first. It's cheaper than either vendor.

Where PandaStack fits (and where it doesn't)

Full disclosure again, since this is my platform: PandaStack is not a GPU cloud. We have no training story, no inference endpoints, no accelerators. If your unit of work is a model forward pass or a fine-tuning run, Modal and RunPod genuinely are the shortlist and we are not on it. I'd rather say that plainly than pretend a comparison exists where one doesn't.

Where we sit is the CPU-side, isolation-first tier of the same stack: hardware-isolated Firecracker microVMs created per task, and git-driven app hosting on the same substrate. There's no warm pool — every create restores a baked snapshot, p50 179ms end to end and p99 around 203ms, with the restore step itself roughly 49ms. Only a template's very first spawn does a real boot, around 3 seconds. Each host pre-allocates 16,384 /30 subnets so network setup never lands in the hot path. You can snapshot a warmed machine and fork it — 400-750ms same-host, 1.2-3.5s cross-host — which is what makes best-of-N agent sampling cheap. And a managed Postgres takes 30-90 seconds to create, because a database bootstrap is a database bootstrap and no amount of snapshot tricks changes that.

The mental model is deliberately the opposite of Modal's. You aren't deploying a function; you're being handed a machine you explicitly drive and explicitly destroy. API surfaces change — check the current docs — but the shape is stable:

from pandastack import Sandbox

# An explicit machine, not a deployed function. Hardware-isolated microVM,
# restored from a baked snapshot: p50 ~179ms to create, no warm pool.
sbx = Sandbox.create(template="code-interpreter", ttl_seconds=600)
try:
    # The model's output goes INTO the guest. The host never executes it.
    sbx.filesystem.write("/work/solve.py", model_generated_code)

    result = sbx.exec("cd /work && python3 solve.py", timeout_seconds=120)
    print(result.exit_code, result.stdout[-500:])

    # Only the artifact crosses back to the trusted side.
    answer = sbx.filesystem.read("/work/out/answer.md")
finally:
    sbx.kill()   # files, processes, network namespace: gone together

Note ttl_seconds. It's a dead-man's switch, because your orchestrator will eventually crash between create and kill, and when it does the VM still has to die. Cleanup code is the least-run and least-tested path in any system, and it is somehow always the one holding the bill — which is, come to think of it, exactly the same lesson as the H100 that stayed warm all weekend.

Pick one

1. Pick Modal if your team is Python-first, your unit of work is a function you wrote, and you want infrastructure to vanish behind a decorator — especially for bursty inference, scheduled pipelines, and wide batch fan-out where a low duty cycle makes per-second billing your friend.
2. Pick RunPod if you're optimising for GPU cost and control at high, sustained utilisation, your environment is too specific for a managed function model, or you want a box you can actually log into while a long training run misbehaves.
3. Pick both, which is more common than the versus framing suggests: rented capacity for the steady training tier, serverless for the spiky inference tier. They're not competing for the same GPU-hour.
4. Pick neither, and go find a sandbox or app platform, if the real problem is untrusted CPU execution with a hard isolation boundary, sub-second creates on an agent's hot path, or a long-lived stateful app with a database — none of which are GPU problems wearing a GPU-shaped hat.

Whatever you pick, go and re-derive the pricing and availability yourself before you commit. This is the fastest-moving corner of the infrastructure market, every number in it has changed at least once since you started reading about it, and the only two figures that reliably matter are your GPU-hours and your duty cycle.