What Is an AI Agent?

An AI agent is a language model running in a loop: the model proposes an action, your code executes that action against the real world, the result is appended to the model's context, and the model proposes again — until it decides it is finished or you stop it. That is the definition. Not a metaphor for it, not a simplification of it. If you strip the vocabulary away from almost every agent shipping today, what remains is a while loop, a list of tool definitions, and a growing list of messages.

I say that up front because the word has been inflated to the point of uselessness. "Agent" now gets attached to anything with an LLM in it, which makes it hard for people building things to figure out what they are actually supposed to build. So this post is the builder-grade version: the loop, the anatomy around it, the difference from a chatbot, the four or five places the loop reliably breaks once real users touch it, and what infrastructure an agent actually needs at runtime.

Disclosure so you can weight the opinions: I run PandaStack, an open-source Firecracker microVM platform, which means I spend my time on the environment layer. I will flag where I'm giving you an opinion rather than a fact. The definition above is vendor-neutral, and so is most of what follows.

The loop is the whole idea

A model on its own produces tokens. Tokens do not clone a repository or query your database. What turns a text generator into something that acts is a very small piece of plumbing: you describe some functions to the model, the model replies with a structured request to call one, and your process actually calls it and hands the result back. Then you ask the model again, with the result now in its context.

Here is that loop with no framework, using the Anthropic Messages API. It is worth writing by hand once, because every agent framework you will ever use is a wrapper around exactly this shape:

import anthropic

client = anthropic.Anthropic()

TOOLS = [
    {
        "name": "read_file",
        "description": (
            "Read a UTF-8 text file from the working directory and return its "
            "contents. Use this to inspect source, config or docs before "
            "answering questions about them. Returns the file text, or an "
            "error string if the path does not exist."
        ),
        "input_schema": {
            "type": "object",
            "properties": {"path": {"type": "string"}},
            "required": ["path"],
            "additionalProperties": False,
        },
    },
]


def step(messages):
    """One turn of the loop: model proposes, we execute, observation goes back."""
    resp = client.messages.create(
        model="claude-opus-5",
        max_tokens=8000,
        tools=TOOLS,
        messages=messages,
    )
    messages.append({"role": "assistant", "content": resp.content})

    if resp.stop_reason != "tool_use":
        return resp, True  # the model answered in prose; the loop is done

    observations = []
    for block in resp.content:
        if block.type != "tool_use":
            continue
        try:
            # YOUR process runs this. The model only asked.
            out = dispatch(block.name, block.input)
            observations.append({
                "type": "tool_result",
                "tool_use_id": block.id,
                "content": str(out)[:20_000],   # cap it: results are history
            })
        except Exception as exc:
            # Hand the failure back as an observation instead of raising.
            observations.append({
                "type": "tool_result",
                "tool_use_id": block.id,
                "content": f"error: {exc}",
                "is_error": True,
            })

    # Every result from one assistant turn returns in ONE user message.
    messages.append({"role": "user", "content": observations})
    return resp, False


messages = [{"role": "user", "content": "What does README.md claim this service does?"}]

for _ in range(20):                 # the iteration cap is not optional
    resp, done = step(messages)
    if done:
        break
else:
    raise RuntimeError("agent hit the iteration cap without finishing")

print(next(b.text for b in resp.content if b.type == "text"))

Look at what is not in there. There is no planner object, no goal decomposition module, no reasoning engine. The "planning" is the model choosing which tool to call next given everything it has seen so far; the "memory" is the messages list; the "autonomy" is the fact that nobody approved the call before dispatch ran it. Those are real properties, but they emerge from the loop rather than being components you install.

Why an agent is not a chatbot

A chatbot is one round trip: text in, text out, the human decides what happens next. An agent closes that decision loop. The differences that follow from that are not cosmetic — they are why agents are harder to build and much harder to operate.

• It acts on the world. A chatbot's worst output is a wrong sentence. An agent's worst output is a wrong sentence that already ran as a shell command.
• It runs multiple turns without you. The whole point is that it acts faster than you can review, which is the feature and the risk in one breath.
• Its own output becomes its input. Tool results feed back into the context, so an error, a hallucination, or an attacker-supplied string compounds across turns instead of being a one-off.
• It has state that outlives a turn. Files it wrote, packages it installed, a server it started. A chatbot has a transcript; an agent has a machine.
• It fails partially. Step four of seven succeeded, step five errored, and now the world is in a half-finished condition that neither the model nor your code has a plan for.

There is also a middle category worth naming honestly, because a lot of shipped "agents" live there: the workflow. If you wrote the sequence of steps and the model only fills in the content of each step, that is an LLM pipeline, and calling it an agent obscures a good design decision. Pipelines are more predictable, cheaper, and easier to debug. My opinion, held firmly: reach for a fixed workflow first, and only give the model control of the sequence when the sequence genuinely cannot be known in advance. Agency is a cost you pay for flexibility, not a badge of sophistication.

The anatomy: four parts, unequal difficulty

Every agent, whatever framework it is dressed in, has four parts. They are not equally hard, and the industry's attention is distributed almost exactly backwards relative to where the work actually is.

• The policy and the loop — the model deciding what to do next, plus the driver code that caps iterations, dispatches calls, and decides when to stop. Difficulty: low. This is the twenty lines above.
• Tools — the functions the model can request, defined by a name, a prose description and a JSON Schema. Difficulty: medium, and almost all of it is writing descriptions clear enough that the model picks correctly. If tool choice is bad, the contract is usually the problem, not the model.
• Memory and context — the messages list, plus whatever you retrieve into it and whatever you compact out of it. Difficulty: medium, and it grows with session length. Note that "memory" means two different things: facts the agent should recall, and the machine state it was working on. They have completely different solutions.
• The environment — the place tool side effects actually happen: a filesystem, a network, a shell, a database, processes that keep running between turns. Difficulty: high, and it is the part that decides whether your agent is safe, whether it is fast, and what it costs to run.

The uncomfortable observation is that the first three are a weekend and the fourth is a company. Prompt engineering has diminishing returns very quickly; the environment does not. I have watched a lot of teams spend three weeks tuning a system prompt and one afternoon deciding to run tool calls in a shared container on the API host, and the second decision is the one that eventually generates the incident review.

Where the loop actually breaks in production

The loop works on your laptop on the first try. Here is what goes wrong once it is running against real inputs, roughly in the order teams discover them.

1. It doesn't terminate. The model calls a failing tool, reads the error, calls it again slightly differently, forever. Without a hard iteration cap you have built a budget-consumption machine. Cap it, log when you hit the cap, and treat hitting it as a bug to investigate rather than noise.
2. Context grows until it is the whole cost. Tool results are conversation history and get re-sent every turn, so one 40,000-row CSV dumped into a result is billed on every subsequent turn. Truncate aggressively, or write big outputs to a file in the environment and return a path plus a summary.
3. An exception escapes the loop. A tool raises, your driver propagates, the conversation dies mid-task and the user gets a 500. Return failures as observations so the model can read them and adapt — that is the behaviour you actually wanted from an agent.
4. The environment is gone. The agent installed a package in turn two and the container that had it was recycled before turn five. Or two concurrent sessions shared one box and turn three read turn one's leftover files. State that lives implicitly on a shared machine is a class of bug that is very hard to reproduce.
5. Half the work happened. The agent created the resource, then errored before recording it. Agents are terrible at compensating transactions on their own, so make tools idempotent where you can and give the agent a way to observe the world rather than assuming its own history is accurate.
6. Something it read told it what to do. Prompt injection is the failure mode with no prompt-level fix: a web page, a PR description, a support email or a previous tool result lands in the same context that chooses the next action, and there is no privileged channel marking some tokens as data. Instructions reduce the rate; they do not change the outcome when one gets through.
7. You cannot tell what happened. Without a log of every tool call, its inputs, its duration and whether it errored, debugging an agent is archaeology. The interesting failure is always a decision three turns before the symptom.

The hard part is the environment, not the prompt

Follow the failure list and you notice that items 4 through 7 are all environment problems wearing different hats. Statefulness, blast radius, injection containment, observability of side effects — none of them are solved by better instructions, and all of them are decided by what kind of machine your tools run on.

This becomes sharpest the moment someone adds a run_code or run_bash tool, which happens on nearly every agent project because it is the highest-leverage tool you can give a model: one tool replaces fifty. It is also the one whose schema is "a string" and whose capability is "whatever this machine can do". Giving an agent code execution is not adding a function. It is handing it a computer.

# pip install pandastack anthropic
from pandastack import Sandbox

RUN_CODE = {
    "name": "run_code",
    "description": (
        "Run Python in an isolated Linux sandbox and return stdout, stderr and "
        "any error. Variables, files and installed packages persist between "
        "calls in the same session, so build work up step by step. Use "
        "print(...) for anything you want to see. The sandbox has no access "
        "to production systems or credentials."
    ),
    "input_schema": {
        "type": "object",
        "properties": {"code": {"type": "string"}},
        "required": ["code"],
        "additionalProperties": False,
    },
}

_sessions = {}


def _context(session_id: str):
    """One sandbox per conversation — not one shared across users.

    Create is ~179ms p50 because it restores a snapshot instead of booting,
    which is what makes per-session freshness affordable rather than a thing
    you avoid by reusing boxes (reuse is where state leaks).
    """
    if session_id not in _sessions:
        sbx = Sandbox.create(
            template="code-interpreter",
            ttl_seconds=1800,        # backstop: it reaps itself if we crash
        )
        _sessions[session_id] = (sbx, sbx.create_code_context())
    return _sessions[session_id][1]


def run_code(session_id: str, code: str) -> dict:
    # This executes inside the guest kernel of a Firecracker microVM.
    # Not on the host running your agent. That distinction is the whole point.
    execution = _context(session_id).run_code(code)
    return {
        "stdout": execution.stdout[-8000:],
        "stderr": execution.stderr[-2000:],
        "error": execution.error,
    }


def end_session(session_id: str) -> None:
    sbx, _ = _sessions.pop(session_id, (None, None))
    if sbx:
        sbx.delete()   # a boundary only helps if you actually destroy it

That is the shape of the answer, and the interesting engineering is entirely below the run_code line: what the boundary is made of, and how cheaply you can throw the environment away. I am not going to re-litigate the isolation ladder here — containers versus microVMs versus full VMs, and what an escape reaches in each — because I wrote that out properly in what is an AI agent sandbox. Read that one for the boundary question. The short version is that the strength you need is a function of who wrote the code, and an agent's code was written by a model.

What an agent needs at runtime

If you are sizing up what to build or buy under an agent, this is the checklist I would use. It is deliberately capability-shaped rather than vendor-shaped.

• A real isolation boundary, strong enough that model-written code escaping it reaches nothing but a throwaway environment.
• Fast, cheap creation. If a fresh environment takes thirty seconds, you will reuse one across sessions, and reuse is how state leaks between users. Latency is a security property here, not just an ergonomic one.
• Statefulness across turns — a filesystem and processes that survive between tool calls, because agents work incrementally and re-installing the world each turn is both slow and confusing to the model.
• Snapshot and fork, so you can checkpoint before a risky step, branch to explore several approaches in parallel, and reset instead of untangling a bad run.
• Egress control: the agent usually needs to fetch packages or data, but "the open internet by default" is how exfiltration and abuse happen. You want an allowlist knob, not a switch.
• Idle-cost near zero. Agent sessions are bursty and mostly idle, waiting on a model or a human, and paying for a warm machine through all that dead time is what makes per-user agents uneconomic.
• Data and long-lived services, because at some point the agent needs a database, a queue, or a web server it started that must still be reachable later.
• Observability of side effects: a log of what ran, where, for how long, and what it touched.

For what it is worth, that list is essentially PandaStack's product spec, which is why I trust it as a checklist and also why you should discount it accordingly. Concretely: every sandbox is its own Firecracker microVM with its own guest kernel; create is around 179ms p50 via snapshot-restore rather than cold boot; a same-host fork lands in roughly 400–750ms, so branching an agent's state is a normal operation rather than a project; managed Postgres, app hosting and cron-scheduled functions cover the long-lived pieces; and everything scales to zero, billed on one rate card at $0.054 per vCPU-hour and $0.0162 per GiB-hour. It is open source, so you can also just read how the boundary is built instead of taking my word for it.

What I'd build first

1. Write the loop by hand, once, with two narrow tools and no framework. You will understand every framework afterwards, and you will stop believing the magic is in the model's planning.
2. Ask honestly whether you need agency at all. If you can write the sequence of steps down, write it down — a workflow beats an agent on cost, latency and debuggability.
3. Spend your prompt effort on tool descriptions rather than the system prompt. They are the only thing the model knows about your system.
4. Cap iterations, return errors as observations, truncate results, and log every call with inputs and duration before you build anything else.
5. Put code execution behind a real boundary before the first demo, not after the first incident. Retrofitting isolation is a rewrite, and you will be doing it under pressure.

An agent is a model that gets to try again. Everything hard about building one comes from that second word: the environment where the trying happens, and what it costs when a try goes wrong.