The anatomy of an AI agent's bill: tokens vs compute vs egress

A founder showed me three browser tabs last month. Tab one: a model provider's usage console. Tab two: a cloud compute bill. Tab three: a spreadsheet where he had been trying, for two weeks, to work out which of the first two tabs was the problem. He had spent those two weeks shaving seconds off sandbox lifetimes. His model bill was roughly thirty times his compute bill. He had optimised the wrong line, thoroughly, and was about to conclude that AI agents were just expensive.

I run PandaStack, a Firecracker microVM platform, so I sell the compute line. Read section two knowing that. I am writing the whole decomposition anyway because I lose more prospective customers to "we optimised the wrong thing and gave up" than I have ever lost to a competitor, and because the honest version of this post is more useful than the flattering one. If your compute bill is small, optimising it is not where your afternoon should go.

What follows is one agent run taken apart into its cost lines, with an order of magnitude and a lever for each. There are no per-token prices in this post and no per-second rates. Those change, sometimes monthly, and a stale price in a blog post is worse than no price. What does not change is the shape of the arithmetic, so I give you the arithmetic and you supply today's numbers off today's pricing pages.

One run, four bills, three vendors, nobody with the total

Start by fixing the unit. Not cost per month, which tells you nothing about whether you are getting more efficient or just growing. Not cost per token, which is the unit your model provider chose for their own convenience. The unit is cost per successful run: one user request, from the moment it arrives to the moment there is an answer good enough to return, including everything you burned on the attempts that did not work.

A run decomposes into four lines:

1. Model tokens. Every input token you send and every output token you get back, across every turn of the agent loop. Billed by your model provider, metered per token, invoiced monthly.
2. Sandbox compute. The seconds a VM or container is alive so the agent has somewhere to run code. Billed by your compute provider, metered per second or per CPU-second and GiB-hour, on a different clock and usually a different invoice.
3. Egress and storage. Bytes the run pulls in and pushes out, plus the snapshots, artifacts and logs it leaves behind. Billed by whoever owns the pipe and the disk, often three different whoevers.
4. The failed run. Not a separate invoice — it is lines one through three paid again for a run that produced nothing, plus the engineer-hours to work out why.

The reason nobody has a single number is structural: three of those lines arrive from different vendors, on different clocks, keyed by nothing in common. Your model provider does not know which customer a request belonged to. Your compute provider does not know which agent run a sandbox served. Fixing that join is the actual first task, and I will come back to it, because every recommendation below is worthless until you can attribute a cost to a run.

Cost per successful run is the only number that goes down when you get better and up when you get worse. Everything else moves when you grow.

Line one: model tokens, and the ratchet nobody prices in

This is usually the biggest line and the one people are least systematic about, for a boring reason: it is invisible. You call agent.run(task) and the loop happens inside somebody's framework. There is no place in your code where you can see the transcript growing, so you reason about token cost as though it were a per-request charge. It is not. It is closer to a compounding one.

The input-token ratchet: input grows quadratically, output linearly

A conversational agent is stateless underneath. The model does not remember turn three when you make turn four; you resend it. Every turn ships the entire prior transcript back over the wire as input tokens. So output tokens grow linearly in the number of turns, and input tokens grow with the square of it.

The arithmetic, once, so you never have to trust an intuition about it again. Let S be the constant prefix — system prompt, tool schemas, whatever repo map or persona you bolt on — resent unchanged every turn. Let G be the average number of tokens one turn adds to the transcript: the assistant's message plus the tool result it triggered. Over N turns, total input tokens are:

total_input  =  sum over k = 1..N of ( S + (k - 1) * G )
             =  N * S  +  G * N * (N - 1) / 2

total_output =  N * A          # A = average assistant message length

Put numbers in it. A coding agent fixing a failing test in a mid-sized Python repo: S = 4,000 tokens of system prompt and nine tool schemas, G = 1,800 tokens per turn of typical assistant message plus typical tool result, A = 350 tokens, N = 12 turns.

• Input: 12 x 4,000 + 1,800 x 66 = 48,000 + 118,800 = 166,800 tokens.
• Output: 12 x 350 = 4,200 tokens.
• Ratio: about 40 input tokens for every output token.

Two things fall out of that immediately. First, if you are reasoning about model cost by thinking about the length of the answers your agent gives, you are looking at 2% of the token volume. Second, the quadratic term means turn count is a superlinear lever. Drop the same run from 12 turns to 8: 8 x 4,000 + 1,800 x 28 = 32,000 + 50,400 = 82,400 input tokens. A third fewer turns cut the input side in half.

Tool output is the multiplier on the ratchet

The G term above is an average, and averages hide the thing that actually bankrupts an agent run: one enormous tool result that then rides along in every subsequent turn's input.

Same run. At turn four the agent runs the test suite with verbose output and gets back 40,000 tokens of pytest chatter — collection lines, a full traceback, a hundred passing test names. That result enters the transcript once. It is then resent as input on turns five through twelve: eight more turns, 40,000 tokens each, 320,000 tokens. The run's input total goes from 166,800 to 486,800. One tool call almost tripled the input side of the entire run.

The model needed maybe 200 tokens of that output: the name of the failing test and the assertion line. You paid for 40,000, nine times over.

The usual offenders, in rough order of how often I see them:

• A headless browser handing back the full DOM or accessibility tree of a real page. Tens of thousands of tokens, of which the agent wanted one button's selector.
• Unfiltered test-runner output. pytest -v, jest verbose, a Go test log with -race chatter. The failing part is usually under 2% of the bytes.
• Package manager logs. An npm install or pip install writes hundreds of progress lines that mean nothing to a model and cost the same as prose.
• A recursive directory listing on a repo with node_modules in it. I have seen this one produce a six-figure token result on its own.
• Whole files read to answer a question about one function. Also: whole CSVs, whole JSON API responses, whole git diffs including the lockfile.
• Base64 image data pasted into a text field because a tool returned it inline.

The fix is three lines of code and it is the single highest-return change in this post, so it gets its own section with a working implementation further down. The principle: the sandbox is a perfectly good place to store bytes. Keep the full output there, give the model a clipped view and a way to ask for more.

Prompt caching moves the price, not the shape

Every major provider now offers some form of caching over a stable prompt prefix, and it is the highest-ratio token lever available because it is a configuration change rather than an architectural one. Mechanically: you mark a prefix as cacheable, the provider keeps the computed state around for a while, and subsequent requests that share that exact prefix are billed at a reduced rate for the cached portion.

The discount, the premium you pay to write the cache, the minimum cacheable size and the time the cache survives all differ by provider and all move. Read the current documentation for the model you are on before you design around it, and re-read it when you switch models.

What does not vary is what breaks it. Caching keys on an exact prefix match, so:

• Anything that mutates the front of the transcript invalidates everything after it. Summarising turns three to six in place, reordering tool definitions, dropping an old message to save space — all of these throw away the cache for the whole rest of the conversation.
• A timestamp, a request id or a random session token in the system prompt makes the prefix unique on every single call. This is the most common self-inflicted cache miss I see, and it is usually one interpolated string.
• Tool schemas that get generated dynamically and serialise their keys in nondeterministic order will silently never hit. Sort them.
• Caching helps the S term the most and the transcript tail the least, so its value goes down as a proportion of the run exactly as the ratchet makes the transcript dominate. It is a large win on short runs and a partial one on long runs.

Design the agent so the transcript is append-only and the prefix is byte-stable, and you get caching for free. Design it so history gets rewritten every few turns for cleverness reasons, and you will pay full price for every token while believing you turned caching on.

Route the loop to a cheap model and the hard step to an expensive one

Look at an agent transcript and count how many turns required intelligence. In most coding-agent runs the answer is one or two. The rest are clerical: read this file, run the tests again, the tests failed so run them with more detail, apply the patch, run them once more. That is dispatch work, and dispatch work does not need your most expensive model.

The pattern is a two-tier loop: a small fast model drives the turns, with an explicit escalation path to a larger one. Escalate on any of these signals, not on vibes:

• The small model called an escalate tool you gave it. Models are surprisingly good at knowing when they are out of their depth if you give them a way to say so.
• The same tool call has now been attempted twice with the same arguments. That is a loop, not progress.
• The task class is known-hard up front — a design decision, an ambiguous bug report, anything where the first turn is planning rather than execution.
• A confidence or self-check step came back negative.

The trap in routing is that it can lose money while looking like it saves it. A cheaper model that takes fifteen turns instead of nine costs more than the expensive one did, because of the ratchet. The quadratic term does not care which model you are paying. So measure routing by cost per successful run, never by cost per token, and if your cheap tier fails more often, count those failures as line four.

Cap the budget, not just the turn count

Every agent needs a hard stop, and the naive one — max_steps — is better than nothing but is the wrong unit. One turn that dumps a 40,000-token DOM is not equivalent to one turn that runs ls. Cap the thing you are actually billed for.

from dataclasses import dataclass, field


@dataclass
class TokenBudget:
    """A hard stop denominated in what you actually pay for.

    max_turns alone lets one fat tool result blow the budget while the
    counter still says "3 of 12". Count tokens; count turns as a backstop.
    """

    max_input: int = 250_000
    max_output: int = 20_000
    max_turns: int = 12

    input_used: int = 0
    output_used: int = 0
    turns: int = 0
    trace: list = field(default_factory=list)

    def charge(self, *, input_tokens: int, output_tokens: int, label: str) -> None:
        self.input_used += input_tokens
        self.output_used += output_tokens
        self.turns += 1
        self.trace.append((self.turns, label, input_tokens, output_tokens))

    def exceeded(self) -> str | None:
        if self.turns >= self.max_turns:
            return f"turn cap: {self.turns}"
        if self.input_used >= self.max_input:
            return f"input cap: {self.input_used:,} tokens"
        if self.output_used >= self.max_output:
            return f"output cap: {self.output_used:,} tokens"
        return None

    def report(self) -> str:
        # The per-turn trace is the point. Sort it and the fat turn is obvious.
        worst = sorted(self.trace, key=lambda r: -r[2])[:3]
        lines = [f"{self.turns} turns  in={self.input_used:,}  out={self.output_used:,}"]
        for turn, label, tin, tout in worst:
            lines.append(f"  turn {turn:>2}  {label:<24} in={tin:,}")
        return "\n".join(lines)


budget = TokenBudget()

while True:
    stop = budget.exceeded()
    if stop:
        # Do not silently truncate. Hand the partial state back to the caller
        # so a human (or a bigger model) can decide whether to continue.
        raise BudgetExhausted(stop, partial=state, trace=budget.trace)

    response = model.complete(messages)
    budget.charge(
        input_tokens=response.usage.input_tokens,
        output_tokens=response.usage.output_tokens,
        label=response.tool_call.name if response.tool_call else "answer",
    )
    ...

print(budget.report())

The report method is the part that earns its keep. Print the three fattest turns of every run in development and you will find your tool-output problem in an afternoon without any instrumentation infrastructure at all.

Line two: sandbox compute, which is mostly not computing

Now the line I sell. The thing to internalise is that a sandbox's cost is driven by the seconds it is alive, and the seconds it is alive are usually much longer than the seconds it computes.

Take the same 12-turn run and time it end to end:

• Restore the sandbox: 0.2 s. On a snapshot-restore platform, create is a restore of a baked snapshot rather than a boot, which is how PandaStack lands at roughly 179 ms p50. On a platform that cold-boots, this line is seconds, and it is on the critical path of every run.
• Waiting on the model: about 42 s across twelve turns, at three and a half seconds a turn. The VM is booted, resident, doing nothing.
• Actually executing commands: 62 s. Test runs, greps, file edits.
• Orchestration overhead, teardown, the bits at the end: 14 s.
• Total wall clock the sandbox exists: about 118 s, of which 42 s — 36% — is the agent thinking.

Thirty-six percent of the compute line, for a VM staring at the ceiling. On a longer chain of reasoning with a slower model it goes past half. This is the single most common waste in agent infrastructure and it is almost never on anyone's list, because it does not look like waste in a trace: the run is making progress, it just is not making progress inside the VM.

There are two meters, and they behave differently

On PandaStack the split is: CPU billed on active CPU-seconds actually burned, memory billed on committed GiB-hours from the moment the VM exists. Other platforms bill full lifetime for both, and some bill a flat instance-second regardless. Check which you are on before optimising, because the correct move differs.

Under the split model, an idle-but-alive sandbox is nearly free on CPU and fully priced on memory. That is a precise and slightly unintuitive statement, so: the 42 seconds of thinking cost you almost no CPU and 42 seconds of 4 GiB of reserved RAM. Idle does not mean cheap. It means paying for the expensive half and getting nothing back.

It also means wall-clock time is the thing to attack, not CPU intensity. Finishing a test suite in 20 s on eight cores instead of 60 s on one core is roughly the same CPU-seconds spread over a third of the GiB-hours. Do not architect around a fear of short bursts. Bursts are the cheap shape.

Do not hold a sandbox open while a model thinks

There are three ways out of the think gap, in descending order of how much they save and ascending order of how much they cost you in complexity.

The first is not to hold a sandbox at all between tool calls. If your tools are stateless shell commands against a filesystem — and for a large fraction of coding agents they are — the sandbox only needs to exist for the duration of the command. Restore it, run the command, kill it. Sub-second restore is what makes this sane: at three seconds a create you would never do it, at a couple of hundred milliseconds it is just how you write the loop, and the compute line drops to the 62 seconds of actual execution.

The honest cost: you lose in-process state. A live Jupyter kernel with a loaded DataFrame, a running dev server, a warm headless browser with a logged-in session — all gone. If your agent depends on any of those, this pattern is wrong for you and you want the second option.

The second is hibernation. Snapshot memory and disk, stop the VM, wake it on the next tool call. State survives, including the in-memory kind, and you stop paying committed memory for the gap. There is a wake cost, so it pays off on gaps of seconds rather than milliseconds — which is exactly the shape of a model call.

from pandastack import Sandbox

sbx = Sandbox.create(
    template="code-interpreter",
    ttl_seconds=900,
    persistent=True,          # hibernation requires a sandbox that outlives a stop
    metadata={"run_id": run_id, "feature": "code-review", "org": customer_id},
)

try:
    for turn in range(MAX_TURNS):
        # ---- the expensive gap: the model is thinking, the VM should not be alive
        sbx.hibernate()
        decision = model.next_action(transcript)     # 3-5 seconds of nothing
        if decision.done:
            break

        # ---- waking is implicit on the next request, but be explicit in a loop
        sbx.wake()
        result = sbx.exec(decision.command, timeout_seconds=120)
        transcript.append(clip(result.stdout, result.stderr))
finally:
    sbx.kill()

The third option, if neither of those fits, is simply to shrink the template. Idle memory is the cost, so less memory is less cost. Which brings up the constraint that surprises people on a snapshot-restore platform.

Memory is a property of the snapshot, not of the request

On a platform where every create restores a baked snapshot, guest RAM is frozen into that snapshot. The VM comes back exactly as large as it was when it was captured, whatever you pass to the create call. There is no per-request memory dial, and that is not an oversight — it is the same property that makes restore fast.

So right-sizing is a per-workload-class decision made once: measure the peak resident set of a real run, add genuine headroom, pick the smallest template above that line, bake one if nothing fits. An oversized template taxes every run forever for memory nobody touches. An undersized one gives you an OOM in the middle of a build, which costs more in engineer-hours than the RAM ever did. The full version of this argument, plus the rest of the compute tactics, is in the sandbox compute bill playbook linked at the bottom.

Fork a warmed snapshot instead of paying setup on every run

The last compute lever is the one that also fixes line three, which is why it is the best-value item in the whole post for anyone running agents at volume.

Look at what a run does before it does anything useful. It boots, installs the same forty packages as last time, downloads the same dataset, warms the same browser, imports the same libraries. On a ninety-second run that can be sixty seconds of setup producing a byte-for-byte identical result to the previous run — paid at full committed memory, plus egress for every byte pulled.

Pay for it once. Build the environment, snapshot it, fork per run. A fork is copy-on-write on both memory and disk, so the child starts from the parent's exact warmed state without redoing any of it; same-host forks land in roughly 400 to 750 ms. Working code is in the fixes section below.

Line three: egress and storage, the line nobody forecasts

This one is small until it is not, and the transition is abrupt. It is invisible in code — one line in a Dockerfile, one from_pretrained call, one logging handler — and it does not show up in the part of the bill anyone is staring at.

What an agent run actually moves over the wire:

• Dependencies on every cold run. pip wheels, npm tarballs, apt packages, a Playwright browser download. A cold Python data-science environment is comfortably in the hundreds of megabytes, per run, if you are not baking or forking.
• Model weights or embeddings pulled into the sandbox. A single from_pretrained on a mid-sized model is gigabytes, and it looks like one line of code.
• Log shipping. Agents are verbose by design and hosted log vendors price per gigabyte ingested. Piping raw exec stdout into your logging pipeline means you pay per token to send it to the model AND per gigabyte to store it.
• Artifacts. A browser agent that screenshots every step produces a surprising amount of PNG. So does a test run that uploads coverage HTML.
• Storage that outlives the compute: snapshots and volumes. Individually tiny, never deleted, and in six months there are eleven thousand of them.

The levers, in order of return:

1. Move the bytes from run time to build time. Bake weights, wheels, browser binaries and caches into the template so they are fetched once at build and are local on every run afterwards. This compounds with forking: a golden sandbox forked a thousand times pulls the dataset once.
2. Pin every version. An unpinned install refetches whenever upstream publishes, silently turning a cached path into an uncached one at a time nobody chose.
3. Stop shipping raw tool output to your log vendor. Ship the clipped version — the same clip you send to the model — plus a pointer to where the full thing lives. You need the full log about one run in fifty.
4. Audit one cold run's network traffic. Watch it from inside the box for a single run. There is always something on the list nobody knew about: a telemetry SDK, a package manager checking for updates, a latest tag re-resolving.
5. Put a retention policy on snapshots and volumes, on a cron, not on a promise.

For most teams this line is the smallest of the four. It gets a section anyway because it is the one with the worst variance: it goes from 3% of the bill to 30% because somebody left a debug flag on, or unpinned a dependency, or added a step that uploads a screenshot. It is a line you monitor rather than a line you optimise.

Line four: the failed run, which pays every other line twice

A run that fails costs the full token bill, the full compute bill and the full egress bill, and produces nothing. Then the retry pays all three again. This is not a separate invoice, which is exactly why it does not appear on anybody's cost dashboard, and it is often the largest single multiplier on the whole thing.

The arithmetic is trivial and worth writing down because it reframes reliability work as cost work:

cost_per_success  =  cost_per_attempt  *  attempts_per_success

attempts_per_success  =  1 / (1 - failure_rate)

  failure_rate 10%  ->  1.11x
  failure_rate 20%  ->  1.25x
  failure_rate 33%  ->  1.50x
  failure_rate 50%  ->  2.00x

And it is worse than that table, for a reason specific to agents: failed runs are systematically longer than successful ones. A run that succeeds does so in six turns. A run that fails flails, retries the same tool call with slightly different arguments, and burns its entire turn budget before hitting the cap. Under the quadratic ratchet, a 12-turn failure costs more than triple a 6-turn success. So a 20% failure rate does not cost you 25% more — it can easily cost you 50% more.

Which means the cheapest cost optimisation available to most teams is not an infrastructure change at all. It is fixing the tool whose error message was so unclear that the model retried it four times. It is adding an example to a tool schema. It is making a flaky test not flaky. None of that looks like cost work, and all of it is.

• Log the turn count of successes and failures separately. If the failure distribution is piled up against your turn cap, your agent is not failing — it is not terminating, which is a different bug with the same invoice.
• Count a run that returned a plausible-looking wrong answer as a failure. It costs the same as a crash and more in trust, and if a human then redoes the work by hand you have paid for the run twice in the most expensive currency there is.
• Track retries per tool, not just per run. One badly specified tool is usually responsible for most of the wasted turns in a whole agent.

The worked decomposition, with symbolic prices

Here is the whole run in one place. Same coding agent, same twelve turns, one fat tool result at turn four, forked from a warmed snapshot so there is no dependency pull. Prices stay symbolic: P_in and P_out per million tokens, C_cpu per vCPU-hour, C_mem per GiB-hour, E per GB egressed.

Line one, model tokens:

• Base input from the ratchet: 12 x 4,000 + 1,800 x 66 = 166,800 tokens.
• The turn-four test dump, resent on turns 5 through 12: 8 x 40,000 = 320,000 tokens.
• Total input: 486,800 tokens = 0.4868 million. Cost: 0.4868 x P_in.
• Total output: 12 x 350 = 4,200 tokens = 0.0042 million. Cost: 0.0042 x P_out.

Line two, sandbox compute:

• Wall clock alive: 118 s. Memory: 4 GiB baked into the template. 4 x 118 / 3600 = 0.131 GiB-hours. Cost: 0.131 x C_mem.
• Active CPU across the run: about 55 CPU-seconds = 0.0153 vCPU-hours. Cost: 0.0153 x C_cpu.
• Of the 118 s, 42 s is model thinking. That fraction of the memory term — 0.047 GiB-hours — is pure waste and is the target of the hibernate pattern above.

Line three, egress and storage:

• Dependency pull: 0 GB, because the run forked a warmed snapshot. On the cold path this would be roughly 0.18 GB per run — nearly two hundred megabytes of wheels and apt packages, on every single run.
• Log shipping, raw exec output: 0.003 GB. Cost: 0.003 x E, plus whatever your log vendor charges per GB ingested, which is usually the bigger half.

Line four, retries: multiply everything above by attempts_per_success. At a 17% failure rate that is 1.2x, and because failures run longer, closer to 1.33x in practice.

Now the only calculation that decides your afternoon:

R  =  token_line / compute_line

   =  (0.4868 * P_in + 0.0042 * P_out)
      -------------------------------------
      (0.131 * C_mem + 0.0153 * C_cpu)

R >> 1   your bill is a model bill. Sections on tokens are your afternoon.
R ~= 1   fix both, starting with whichever has the cheaper lever.
R << 1   your bill is a compute bill. Go read the compute playbook.

I am not going to tell you what R is, because that requires prices I refuse to put in a blog post that will still be indexed in eighteen months. But I will tell you the shape: the numerator is denominated in hundreds of thousands of tokens and the denominator in hundredths of a vCPU-hour. For a text-heavy agent driving a frontier model over a handful of shell commands, R comes out well above one, frequently by an order of magnitude. That is not a claim about any specific vendor's rates; it falls out of the ratchet. Half a million input tokens is a lot of tokens, and two minutes of a 4 GiB VM is not a lot of VM.

R inverts, and the whole priority order in this post flips with it, when the run does real work:

• A twenty-minute build or test suite driven by three model turns. The compute line is two orders of magnitude bigger than in the example above and the token line barely moved.
• A browser farm where each run drives a headless Chrome for ten minutes. Memory-heavy, long-lived, chatty on egress.
• Video or media processing, ML training or fine-tuning, anything with a GPU attached. The token line becomes a rounding error.
• An agent on a small self-hosted open-weights model, where P_in is your own compute cost rather than a provider's list price.

Here is the estimator as a script. Every rate is zero on purpose: fill them in from current pricing pages, put the date in a comment, and re-run it when you change models.

#!/usr/bin/env python3
"""Decompose one agent run into its four cost lines.

EVERY rate below is a PLACEHOLDER set to zero. Fill them from the pricing
pages you are actually on, on the day you read them, and note the date.
The workload numbers are from one instrumented run -- replace them with
yours. The output number is not the point; the SHARES are.
"""

# ---- RATES: fill these in (and write down the date you read them) --------
PRICE_IN_PER_MTOK    = 0.0   # model provider, input, per 1,000,000 tokens
PRICE_OUT_PER_MTOK   = 0.0   # model provider, output, per 1,000,000 tokens
PRICE_CACHED_PER_MTOK= 0.0   # cached-read input rate, if your provider has one
PRICE_CPU_PER_HR     = 0.0   # compute provider, per vCPU-hour
PRICE_MEM_PER_GIB_HR = 0.0   # compute provider, per GiB-hour
PRICE_EGRESS_PER_GB  = 0.0   # network egress, per GB
PRICE_LOG_PER_GB     = 0.0   # log vendor, per GB ingested

# ---- WORKLOAD: measure these on one real run -----------------------------
TURNS                = 12
PREFIX_TOKENS        = 4_000     # system prompt + tool schemas, resent each turn
GROWTH_PER_TURN      = 1_800     # typical assistant msg + typical tool result
ASSISTANT_TOKENS     = 350       # average output per turn
FAT_RESULT_TOKENS    = 40_000    # the one huge tool result...
FAT_RESULT_AT_TURN   = 4         # ...and the turn it lands on
CACHED_FRACTION      = 0.0       # share of input served from prompt cache

WALL_SECONDS         = 118.0     # sandbox alive, create -> kill
MODEL_WAIT_SECONDS   = 42.0      # of which: waiting on the model
CPU_SECONDS          = 55.0      # active CPU actually burned
MEM_GIB              = 4.0       # baked into the template snapshot

EGRESS_GB            = 0.0       # deps pulled at run time (0 if you fork)
LOG_GB               = 0.003     # exec output shipped to your log vendor

FAILURE_RATE         = 0.17      # share of runs that produce nothing usable
FAILURE_LENGTH_RATIO = 1.6       # failures run longer than successes


def input_tokens() -> int:
    """N*S + G*N*(N-1)/2, plus a fat result riding along to the end."""
    base = TURNS * PREFIX_TOKENS + GROWTH_PER_TURN * TURNS * (TURNS - 1) // 2
    rides_for = max(0, TURNS - FAT_RESULT_AT_TURN)
    return base + FAT_RESULT_TOKENS * rides_for


tin = input_tokens()
tout = TURNS * ASSISTANT_TOKENS
cached = tin * CACHED_FRACTION
fresh = tin - cached

tokens = (
    fresh / 1e6 * PRICE_IN_PER_MTOK
    + cached / 1e6 * PRICE_CACHED_PER_MTOK
    + tout / 1e6 * PRICE_OUT_PER_MTOK
)
compute = (
    CPU_SECONDS / 3600.0 * PRICE_CPU_PER_HR
    + MEM_GIB * WALL_SECONDS / 3600.0 * PRICE_MEM_PER_GIB_HR
)
bytes_line = EGRESS_GB * PRICE_EGRESS_PER_GB + LOG_GB * PRICE_LOG_PER_GB

attempt = tokens + compute + bytes_line
attempts_per_success = 1.0 / (1.0 - FAILURE_RATE)
waste_multiplier = 1.0 + (attempts_per_success - 1.0) * FAILURE_LENGTH_RATIO
success = attempt * waste_multiplier

idle_mem = MEM_GIB * MODEL_WAIT_SECONDS / 3600.0 * PRICE_MEM_PER_GIB_HR

print(f"input tokens        : {tin:>12,}")
print(f"output tokens       : {tout:>12,}")
print(f"input:output ratio  : {tin / max(tout, 1):>12.1f} : 1")
print()
print(f"1. tokens           : {tokens:>12.6f}")
print(f"2. compute          : {compute:>12.6f}   (idle share {idle_mem:.6f})")
print(f"3. egress + logs    : {bytes_line:>12.6f}")
print(f"   cost / attempt   : {attempt:>12.6f}")
print(f"4. retry multiplier : {waste_multiplier:>12.2f}x")
print(f"   COST / SUCCESS   : {success:>12.6f}")
print()
if compute > 0:
    print(f"R (tokens/compute)  : {tokens / compute:>12.1f}")

# Now run the three experiments that actually decide your roadmap:
#   1. FAT_RESULT_TOKENS = 2_000   -> what clipping tool output is worth
#   2. TURNS = 8                   -> what a tighter loop is worth (quadratic!)
#   3. WALL_SECONDS -= MODEL_WAIT_SECONDS -> what hibernating the think gap is worth
# Whichever delta is biggest is your next sprint. The other two can wait.

Run those three experiments before you write any code. On the parameters above, experiment one — clipping the fat tool result from 40,000 tokens to 2,000 — removes 304,000 input tokens from a 486,800-token run. That is a 62% cut to the largest line in the decomposition, from a change you can make before lunch. Experiment three removes 36% of the memory term from a line that was already smaller. This is precisely the prioritisation the founder in the opening paragraph did not have.

The two fixes with the best return, in full

Fix one: clip tool output before it re-enters the context

The rule: the model gets a view, the sandbox keeps the bytes. Head and tail beat a plain truncate, because the interesting part of a failure is usually at the end and the interesting part of a listing is usually at the start. Always tell the model what you elided and how to get it, or it will re-run the command to see the rest, which costs you a turn and the ratchet costs you the rest.

import hashlib
import re

MAX_CHARS = 6_000          # roughly 1,500 tokens; tune per tool, not globally
HEAD_SHARE = 0.35          # start of output: setup, collection, the command echo


def clip(text: str, *, path: str, max_chars: int = MAX_CHARS) -> str:
    """Return a head+tail view of `text`, with a pointer to the full copy.

    `path` is a file INSIDE the sandbox where the caller has already written
    the complete output. The model can grep it if it wants more. This is the
    single highest-return change in the whole cost decomposition: a result
    clipped once is a result you are not resending on every subsequent turn.
    """
    if len(text) <= max_chars:
        return text

    head_n = int(max_chars * HEAD_SHARE)
    tail_n = max_chars - head_n
    elided = len(text) - max_chars
    digest = hashlib.sha256(text.encode()).hexdigest()[:12]

    return (
        f"{text[:head_n]}\n"
        f"\n[... {elided:,} characters elided. Full output ({len(text):,} chars, "
        f"sha {digest}) is at {path} inside the sandbox. "
        f"Use grep or sed on that path to see more; do not re-run the command. ...]\n"
        f"\n{text[-tail_n:]}"
    )


# --- better still: know what the tool is, and keep only the useful part ----

PYTEST_SUMMARY = re.compile(
    r"(^=+ (FAILURES|ERRORS) =+$.*?)?^=+ .*(failed|passed|error).*=+$",
    re.MULTILINE | re.DOTALL,
)


def clip_pytest(text: str, *, path: str) -> str:
    """A test runner has a known shape. Exploit it.

    Passing test names are worth nothing to the model and cost the same as
    prose. The failure block and the summary line are worth everything.
    """
    match = PYTEST_SUMMARY.search(text)
    if not match:
        return clip(text, path=path)
    return clip(match.group(0), path=path, max_chars=8_000)


# --- wiring it into the loop ----------------------------------------------

def run_tool(sbx, command: str, turn: int) -> str:
    out_path = f"/tmp/agent-out/turn-{turn:02d}.log"
    sbx.exec(f"mkdir -p /tmp/agent-out && {command} > {out_path} 2>&1; true")
    full = sbx.filesystem.read(out_path).decode("utf-8", "replace")

    if command.startswith(("pytest", "python -m pytest")):
        return clip_pytest(full, path=out_path)
    return clip(full, path=out_path)

Two details that matter more than they look. The output never transits your orchestrator at full size — it is written inside the sandbox and only the clipped view crosses the wire, so you are not paying egress to move 40,000 tokens of pytest output out of a VM just to throw it away. And the elision marker explicitly tells the model not to re-run the command, which is the failure mode that turns a saved 38,000 tokens into a wasted turn plus 38,000 tokens anyway.

Give the model a grep tool over that directory and it will use it well: a targeted grep against a 40,000-token log costs a few hundred tokens and answers the question. That is the whole trick — replace a push of everything with a pull of the relevant part.

Fix two: fork a warmed snapshot instead of a cold install

This one attacks the compute line and the egress line at the same time, and it changes the run's shape rather than shaving it. Build the environment once, snapshot it so it survives a restart, then fork a child per run. Copy-on-write memory and reflinked disk mean the child inherits the parent's warmed state without redoing any of it.

from pandastack import Sandbox

GOLDEN_TTL = 3600


def build_golden() -> Sandbox:
    """Pay the expensive setup exactly once. Runs in CI, not in the hot path."""
    golden = Sandbox.create(
        template="base",
        ttl_seconds=GOLDEN_TTL,
        persistent=True,
        metadata={"env": "build", "feature": "golden-image", "owner": "team-agents"},
    )

    # Everything a run would otherwise redo on every single invocation:
    golden.exec("pip install -r /work/requirements.txt", check=True)
    golden.exec("python -c 'import pandas, numpy, transformers'", check=True)  # warm imports
    golden.exec("python /work/prefetch_tokenizer.py", check=True)              # weights local
    golden.exec("git -C /work/repo fetch --depth 50 origin", check=True)

    # A durable copy, so the golden box is reproducible after a deploy.
    golden.snapshot()
    return golden


def run_task(golden: Sandbox, task: str, run_id: str) -> str:
    """Per run: no install, no download, no import warm-up, no egress."""
    child = golden.fork(metadata={"run_id": run_id, "env": "prod"})
    try:
        # Same-host forks land in roughly 400-750 ms and start from the
        # parent's exact state -- packages installed, caches hot, repo fetched.
        return child.exec(f"python /work/agent.py {task}", timeout_seconds=600).stdout
    finally:
        child.kill()


golden = build_golden()
for run_id, task in queue:
    print(run_task(golden, task, run_id))

Two warnings from getting this wrong. The golden sandbox is now infrastructure with a lifecycle: it needs rebuilding when dependencies change, and that rebuild belongs in CI rather than in somebody's head. And a fork inherits everything, secrets included. Build the golden box from a clean state with no credentials in it and inject per-run secrets into the child. A fork is an extremely efficient way to copy a mistake a thousand times.

Instrument the run, not the month

Everything above depends on being able to attribute cost to a run, and that join does not exist by default because the three vendors involved share no key. You have to create one and carry it through every call. It is not hard; it is just something nobody does until the bill is already confusing.

Emit one record per run with all four lines in it:

import json
import time
import uuid
from dataclasses import asdict, dataclass, field

from pandastack import Sandbox


@dataclass
class RunLedger:
    """One row per agent run. This is the join key across three vendors."""

    run_id: str = field(default_factory=lambda: str(uuid.uuid4()))
    org: str = ""
    feature: str = ""
    started_at: float = field(default_factory=time.time)

    # line 1 -- from your model client's usage object, every call
    turns: int = 0
    input_tokens: int = 0
    cached_input_tokens: int = 0
    output_tokens: int = 0
    model_ids: list = field(default_factory=list)

    # line 2 -- from the sandbox lifecycle
    sandbox_ids: list = field(default_factory=list)
    wall_seconds: float = 0.0
    model_wait_seconds: float = 0.0      # the idle you can still remove
    cpu_seconds: float = 0.0
    mem_gib: float = 0.0

    # line 3
    egress_bytes: int = 0
    log_bytes: int = 0

    # line 4
    outcome: str = "unknown"             # success | failed | capped | error
    attempt: int = 1
    stop_reason: str = ""

    def emit(self) -> None:
        row = asdict(self)
        row["duration_s"] = round(time.time() - self.started_at, 3)
        row["idle_fraction"] = (
            round(self.model_wait_seconds / self.wall_seconds, 3)
            if self.wall_seconds else 0.0
        )
        print(json.dumps(row))          # ship it wherever your analytics live


# The same run_id goes into sandbox metadata, so the compute side of the
# join exists too. Without this, sandbox spend is unattributable the moment
# the VM stops existing -- there is nothing left to tag retroactively.
ledger = RunLedger(org=customer_id, feature="code-review")
sbx = Sandbox.create(
    template="base",
    ttl_seconds=900,
    metadata={"run_id": ledger.run_id, "org": customer_id, "feature": "code-review"},
)

From that one row you can build the only two dashboards worth having. The first is cost per successful run, sliced by feature and by customer, plotted over time. It is the number that tells you whether engineering effort is working. The second is the four-line split as a stacked share of that cost, which tells you where to point the effort next month.

Note the deadline embedded in this. Attribution happens at create time or not at all. You can retroactively re-tag a storage bucket; you cannot retroactively label a VM that stopped existing four hours ago, because there is nothing left to attach the label to. Every week you run without a run_id in sandbox metadata is a week of spend you will never be able to explain, and therefore never cut.

When the compute line is not worth your afternoon

The uncomfortable corollary of everything above: if R is 20, then a heroic 40% cut to your compute line is a 2% cut to your total. That is not a rounding error worth a sprint. It is a rounding error worth a note in a backlog you never look at.

So, plainly, the cases where a microVM sandbox platform is the wrong answer to a cost question:

• Your agent never executes code. It calls a search API, a CRM, and a summariser. There is no compute line. Everything in section two is noise for you, and buying a sandbox platform to fix your bill would be buying a solution to a problem you do not have.
• You run one long-lived, steadily-loaded service rather than bursty per-request work. Per-second microVMs are priced for bursts and idleness. A reserved instance or a container platform with committed-use discounts will beat us on a workload that is busy 24 hours a day, and I would rather tell you that than have you find out in month three.
• You are running ten agent runs a day. Your bill is not a problem yet and any hour spent on it is an hour not spent on the product. Come back at a thousand a day. If you buy isolation from us at that volume, buy it for the KVM boundary and the blast radius, not for the price.
• Your dominant cost is a GPU. Nothing in this post moves that needle; go read about GPU utilisation and batching instead.

And the case where the compute line genuinely is the answer: your runs execute real work — builds, test suites, browser sessions, data processing — for minutes at a time, at volume, with long idle gaps while a model thinks in between. That shape is where sub-second restore, copy-on-write forking and scale-to-zero stop being nice properties and start being the difference between a viable unit economic and an unviable one. It is also, not coincidentally, the shape we built for.

If your compute bill is small, optimising it is not where your afternoon should go. I sell compute and I still think that is the most useful sentence in this post.

The order I would attack it in

1. Instrument one run end to end. Four numbers: input tokens, output tokens, sandbox wall seconds, bytes moved. One afternoon, and every decision after it is grounded.
2. Compute R. Divide the token line by the compute line. This single division tells you which two-thirds of this post to ignore.
3. If your failure rate is above about 10%, fix that first regardless of R. It is a multiplier on all three other lines and it is usually one bad tool schema.
4. Clip tool output. Highest return per line of code in the whole list, and it improves agent quality at the same time — models reason worse with 40,000 tokens of pytest noise in the window, not just more expensively.
5. Turn on prompt caching for a byte-stable prefix. A configuration change, not an architecture change. Verify it is actually hitting; the most common outcome is a cache that never hits because of an interpolated timestamp.
6. Cap the budget in tokens, not turns, and print the three fattest turns of every run in development.
7. Route the loop to a smaller model, measured strictly by cost per successful run, and be willing to revert it.
8. Only now, compute: kill or hibernate the sandbox during model calls, fork a warmed snapshot instead of reinstalling, right-size the template once per workload class.
9. Egress: bake dependencies into the template, pin versions, stop shipping raw output to your log vendor.
10. Re-measure. Then set a monthly alert on cost per successful run so that the next regression finds you rather than the other way round.

None of that is clever. It is measurement, then arithmetic, then attacking the biggest number — which has been the correct approach to every infrastructure bill for thirty years. It stays worth writing down because agents make the measurement step genuinely hard: the costs land on three invoices from three vendors with no shared key, and the largest one grows with the square of a variable nobody is watching.

Get the four numbers. Do the division. Then go and fix the line that is actually big.