Agent tool permissions, explained

A tool call went through that should not have. Maybe it was a delete. Maybe it was a query against a table the person asking had no business reading. You go looking for the control that should have caught it, and what you find is a line in the system prompt: never modify production records without confirmation. That line is not a control. It is a hope with good grammar.

There is one reframe that fixes the whole design, and it is smaller than it sounds. The model is not a principal.

Every permission system that has ever worked authorizes a principal — a user, a service account, a device — to take an action on a resource. When you hand an agent six tools, the intuitive move is to treat the model as that principal: the agent may search, may read files, may send mail. But a model's behaviour is a function of its input, and its input includes text from the outside world. A fetched web page. An uploaded PDF. A support ticket filed by a stranger. A principal whose decisions can be authored by an attacker is not a principal at all. It is an input.

So permissions attach to the session — which human is this agent acting for, right now — and they get enforced below the model, in code the model cannot talk its way past. I build PandaStack, a Firecracker microVM sandbox platform, so the infrastructure section below has an obvious bias. The rest is provider-independent and mostly framework-independent too.

Who the principal actually is

The principal is the end user on whose behalf the agent is running. Not the agent. Not your application. The specific human whose request started this turn.

The most common design error follows directly from getting this wrong: the agent process holds one service-account key with broad rights, every tool uses it, and the per-user check happens nowhere. It feels fine in development, because in development you are the only user and you have every right anyway. It stops being fine the moment a second user exists.

Say it plainly. If the agent can do more than the user who asked, you have built a privilege escalation machine, and the exploit is asking nicely.

The fix is unglamorous. Mint a credential per session from the identity of the user who authenticated, scoped to what that user can do, expiring on a timescale close to the length of a run. Most identity providers support some form of token exchange for exactly this. If your data layer cannot issue per-user credentials, pass a principal object into every tool and have the data layer filter on it — in code, server side, never in a string the model helped assemble.

The failure this prevents is quieter than a rogue delete. Take retrieval over a document store whose index is not access-filtered. A user asks a broad question, the retriever returns the three most relevant chunks, and one comes from a document that user cannot open. The model summarises it faithfully. No tool was misused, no rule was broken, and no log will record it as an incident. You leaked the document anyway.

Where enforcement lives

A rule can live in three places in an agent system. Only two are enforcement.

The prompt is not one of them. A system prompt instruction is a strong prior on behaviour and nothing more. It holds until a user writes a persuasive message, until a fetched page carries a more assertive instruction than yours, or until the model reasons its way to an exception it finds convincing. Tool descriptions are worse: they read like documentation and are in fact advertising copy aimed at the model. Prompt injection is only the sharp end of this. Ordinary user confusion gets past prompt rules several times a week.

Keep the prompt line anyway. It lowers the rate at which the model tries, which shows up as fewer denials and fewer dead ends for legitimate users. That is defense in depth, and the phrase is exact: a layer that reduces frequency, not one that changes outcomes.

Enforcement lives in the tool implementation and in the infrastructure underneath it. That is it. A useful test: can you write an automated test where the model asks for the forbidden thing and the code refuses? If the answer involves checking whether the model complied, you do not have a control — you have a behaviour you are monitoring.

The confused deputy problem

This is the failure mode that makes agents different from ordinary software, and it has a name that predates all of this by decades. A deputy holds authority on someone's behalf and is tricked into using it for someone else's purpose. Your agent holds the user's authority. The purpose can arrive from anywhere it reads.

The chain is short. The agent fetches a page, opens an uploaded file, or reads an incoming email. That content contains text shaped like an instruction. The model, with no reliable way to separate the user's intent from a document's contents, calls a privileged tool. The privilege was the user's. The intent came from a stranger.

Three mitigations, in order of effect.

Separate the reading context from the acting context. Have a summariser with no tools at all ingest the untrusted document and return structured data — extracted fields, a classification, a summary. The acting agent sees that as data, in a slot you control, not as free text in its instruction stream. It is more work, and it is the most effective item on this list.

Downgrade privileges after ingestion. Mark the session tainted the moment untrusted content enters the context, and let that flag gate every write-shaped tool. A run that has read the open web can still query, still compute, still draft — it just cannot send, delete, or transfer without coming back to the user.

Require attribution for the acting step. Before a write executes, you should be able to point at the originating user turn and say: this is the request that authorised this. When the honest answer is that a fetched page asked for it, the call should stop.

from dataclasses import replace

READ_ONLY_TABLES = frozenset({"orders", "customers", "invoices"})

def fetch_page(session: Session, url: str) -> tuple[str, Session]:
    """Reading the outside world is where trust ends."""
    text = http_get(url).text[:20_000]
    # Everything downstream of this call has been influenced by a stranger.
    tainted = replace(
        session,
        tainted=True,
        allowed_tables=session.allowed_tables & READ_ONLY_TABLES,
    )
    return text, tainted

def send_email(session: Session, to: str, body: str) -> dict:
    if session.tainted:
        raise Denied(
            "write tool called in a context that ingested untrusted content; "
            "needs explicit confirmation from the user who started this turn"
        )
    if to.split("@")[-1].lower() not in session.allowed_recipient_domains:
        raise Denied("recipient domain not permitted for this session")
    return mailer.send(from_user=session.user_id, to=to, body=body)

Taint tracking is coarse and it will annoy people. One fetch downgrades a whole run, including the parts that had nothing to do with the fetched content. Start narrow: gate only the tools that change state or leave your system, and widen it if something slips past. A permission model nobody can work with gets switched off, which is worse than a modest one that stays on.

Granularity that survives contact

Most agent permission schemes stop at the tool boundary. This agent may call search, that one may not send email. It looks like RBAC, it fits neatly in a config file, and it is nearly useless — because the interesting variable is almost never which tool, it is which arguments.

Sending email is fine. Sending email to an address the model found in a fetched page is not. Running SQL is fine. Running a DELETE with no WHERE clause is not. Reading a file is fine. Reading a file from a path the model constructed by joining a filename it read out of a document is how you end up serving the contents of a private key.

So the real checks are parameter-level, and they belong in the tool implementation rather than the schema. A tool's JSON schema is a hint to the model about call shape. Nothing validates it on the way in unless you do, and a model can emit whatever it likes. Treating it as a security boundary is the same mistake as trusting a disabled button in a web form.

import json, logging, time
from dataclasses import dataclass

audit = logging.getLogger("agent.audit")

class Denied(Exception):
    pass

@dataclass(frozen=True)
class Session:
    """The principal, resolved once when the run starts — never from model output."""
    user_id: str
    org_id: str
    turn_id: str                    # the user message that started this run
    allowed_tables: frozenset[str]  # derived from the user's own grants
    allowed_recipient_domains: frozenset[str]
    tainted: bool = False

def query_table(session: Session, table: str, limit: int = 100) -> dict:
    """Tool: query_table. The model chooses the table. It does not choose access."""
    decision, rule = "allow", "table_allowlist"
    try:
        if table not in session.allowed_tables:
            decision, rule = "deny", "table_not_in_session_allowlist"
            raise Denied("no access to table " + table)

        # Tenant scoping happens in the data layer, from the session — not
        # from anything the model produced. There is no argument for org_id.
        rows = db.select(table, org_id=session.org_id, limit=min(limit, 1000))
        return {"rows": rows}
    finally:
        audit.info(json.dumps({
            "ts": time.time(),
            "principal": session.user_id,
            "org": session.org_id,
            "originating_turn": session.turn_id,
            "tool": "query_table",
            "params": {"table": table, "limit": limit},
            "tainted_context": session.tainted,
            "decision": decision,
            "rule": rule,
        }))

Two details in there do most of the work. There is no org_id parameter, so the model cannot supply one; tenant scoping comes from the session and the data layer applies it. And the allowlist is an allowlist. Denylists lose against a generator that produces novel strings all day — you will block DROP and get dropped by TRUNCATE.

Boundaries you do not have to reason about

Everything so far is code you must get right on every path, in every tool, forever. That is why the infrastructure layer matters: every permission you can enforce structurally is one you do not have to remember.

Three of these pay for themselves quickly. Egress policy per execution environment, so code the agent wrote reaches only the hosts you named — that alone turns most exfiltration into a failed connection. No ambient cloud credentials where model-written code runs: no instance metadata endpoint, no mounted service account file, no API keys in the process environment where one print statement reveals them. And a filesystem belonging to a disposable machine, so persistence has nowhere to live.

This is where PandaStack fits, and it is worth being precise about what it does not solve. A sandbox does not stop a bad query; that query reaches your database, and your tool-level check is the only thing between them. What it stops is a bad outcome becoming a foothold. Each sandbox gets its own network namespace, an ephemeral root filesystem, and a TTL that reaps it whether or not your code remembered to. Creation runs about 179ms at p50 on the snapshot-restore path, which matters for a boring reason: when a fresh isolated environment costs less than a page load, per-task isolation becomes the default rather than something you reserve for risky work.

from pandastack import Sandbox

# Inputs go in as data. Credentials do not go in at all — the guest has no
# cloud identity, so there is nothing for generated code to find and use.
sbx = Sandbox.create(
    template="base",
    ttl_seconds=300,
    metadata={"user": session.user_id, "turn": session.turn_id},
)
try:
    sbx.filesystem.write("/work/input.json", json.dumps(payload))
    r = sbx.exec("python /work/analyse.py")
    result = r.stdout[:8000]   # cap what re-enters the model's context
finally:
    sbx.kill()

Tagging the environment with the principal and the originating turn is what lets you answer, three days later, which user's run produced the artifact you are staring at.

If you cannot say who caused this write

A permission model you cannot audit is one you cannot debug, and one you will eventually stop trusting. Each tool call needs four things recorded: the principal, the tool and its parameters, the originating user turn, and the decision plus the rule that made it.

The originating turn is the field everyone forgets and the one that matters during an incident. It is how you tell the user asked for this apart from a page the agent read asked for this. Without it you have a list of things that happened, in order, with no causal structure.

Log denials as loudly as allows. Denials are the interesting rows: a spike is either an attack or a broken permission grant, and you want to hear about both within minutes. A permission system with no denials in its log is either unused or not wired up, and you cannot tell which from outside.

One honest concession before you build any of this. For an internal tool with one user — you — a properly scoped token and a disposable execution environment already give you most of the safety on this page, and a policy engine gives you configuration to maintain and nothing else. Do not build RBAC for one person. Keep the audit log anyway: it is the cheapest thing here, and it is how you find out what your agent does when you are not watching.

The rest, in order of what to do first:

1. Resolve the principal once, when the run starts, from authentication — never from anything the model produced.
2. Scope the credential to that user. If the agent can do more than the asker, fix that before anything else on this list.
3. Move every rule that matters out of the prompt and into the tool body.
4. Check parameters, not just tool names, with allowlists rather than denylists.
5. Taint the session on untrusted ingestion and gate write-shaped tools on it.
6. Run code in an isolated environment with no ambient credentials and restricted egress.
7. Log principal, parameters, originating turn, decision, and rule — for allows and denials alike.

None of this makes the model trustworthy, and that is the point. The model stays what it is: a very capable component that reads text from strangers. You put the authority somewhere else.