How to Build an Agent Loop From Scratch (It's About 100 Lines)

The loop itself is trivial. Everything around it is not.

An agent loop is four steps:

1. Send the conversation plus your tool definitions to the model
2. Check the response for tool calls
3. Execute the ones it asked for and append the results to the conversation
4. Send it back, and repeat until the model replies without asking for tools

That is genuinely all of it — about 100 lines of ordinary Python. You do not need a framework, and you certainly do not need one before you understand this much.

What you do need is the other four things, and they are what every tutorial leaves out.

Thing 1 — tool errors must go to the model, not up the stack

The obvious implementation calls your function and lets exceptions propagate. That ends the run.

The better one catches the exception and returns it to the model as a string:

def run(self, **kwargs) -> str:
    try:
        result = self.fn(**kwargs)
    except Exception as exc:
        return f"ERROR: {type(exc).__name__}: {exc}"

A model told "ERROR: KeyError: No invoice INV-999. Known: INV-001, INV-002" will usually correct its argument and try again. A model that never hears about the error cannot.

Thing 2 — the provider difference that breaks multi-provider code

This is the one that costs people an afternoon.

After you execute tools, Anthropic expects one user message containing every tool result:

{"role": "user", "content": [
    {"type": "tool_result", "tool_use_id": "tu_1", "content": "..."},
    {"type": "tool_result", "tool_use_id": "tu_2", "content": "..."}
]}

OpenAI expects one message per result, each with a tool role:

{"role": "tool", "tool_call_id": "call_1", "content": "..."}
{"role": "tool", "tool_call_id": "call_2", "content": "..."}

Getting this wrong produces a confusing 400 rather than a helpful error. The clean fix is to have your result builder return a dict for one provider and a list for the other, and let the loop append or extend accordingly.

There is a second difference worth knowing: OpenAI returns tool arguments as a JSON string that can arrive malformed, so parse it defensively. Anthropic returns a parsed object.

Thing 3 — cost tracking that matches the bill

Do not count tokens locally. Local estimates drift from what you are charged, because system prompts, tool definitions and cached content all count in ways that are tedious to reproduce and change without notice.

Read the usage block the provider returns with each response. It is authoritative because it is what the billing system used.

Then handle the part most trackers miss: cached tokens are billed at different rates — roughly 10% for reads and 125% for writes. Counting them as ordinary input tokens under-reports on any long conversation, which is exactly where costs actually accumulate.

One design rule worth adopting: if you do not recognise the model, report zero rather than guessing. A wrong number is worse than a missing one, because wrong numbers get trusted and end up in somebody's margin calculation.

Thing 4 — retries that respect the server

When you hit a rate limit, the provider usually tells you how long to wait in a retry-after header. Use it. Backing off on a schedule the server did not ask for turns a rate limit into a longer one.

Where there is no header, use exponential backoff with full jitter:

time.sleep(min(2 ** attempt + random.random(), 30.0))

The random component is not decoration. Without it, every client that hit the same rate limit retries at the same instant and hits it again together — a thundering herd that turns one bad second into a bad minute.

The two guards you want before this touches a customer

A turn limit. Models can loop. Cap the number of round trips and return a clear message when you hit it, rather than running until something else breaks.

A budget. Check accumulated cost before each API call and stop if the run would exceed it. This is the difference between a bug costing forty cents and costing four hundred dollars overnight.

Neither is exotic. Both are missing from almost every example agent on the internet.

On streaming

Streaming and tool calling are separate protocols on both providers, and combining them is genuinely fiddly — you are reassembling partial JSON from deltas.

Keep them apart. Use the loop for tool work, and a separate streaming path for the conversational replies where time-to-first-token actually matters. Mixing both into one method is how these codebases become unreadable.

Do you need a framework?

Eventually, maybe. Not to start.

Frameworks earn their place when you need their specific abstractions — retrieval pipelines, complex multi-step chains, a plugin ecosystem. They cost you when you do not, because they hide the wire format, and the wire format is precisely what you need to see when a tool call comes back malformed at 2am.

Write the 100 lines first. Adopt a framework when something concrete demands it, not in anticipation.

Frequently asked questions

How do I build an AI agent loop from scratch?

Send the conversation and your tool definitions to the model, check the response for tool calls, execute the requested ones, append the results to the conversation, and send it back. Repeat until the model replies without asking for tools or a turn limit stops it. That is about 100 lines of Python. The rest of a production agent — budgets, retries, cost accounting and turn limits — exists to make those lines safe to run for a paying customer.

What is the difference between Anthropic and OpenAI tool calling?

After executing tools, Anthropic expects a single user message containing every tool result, while OpenAI expects one message per result with a tool role. Anthropic returns tool calls as content blocks of type tool_use with parsed input; OpenAI returns them under message.tool_calls with arguments as a JSON string that can arrive malformed and needs defensive parsing. That single structural difference is what breaks most multi-provider agent code.

How do I track LLM API costs accurately?

Read the usage block returned with each response rather than counting tokens locally, because local estimates drift from the bill — system prompts, tool definitions and cached content all count in ways that change without notice. Handle cached tokens separately, since they are billed at roughly ten per cent for reads and one hundred and twenty-five per cent for writes. If a model is unrecognised, report zero rather than guessing.

Do I need LangChain to build an agent?

No. A tool-calling loop is about 100 lines against the HTTP API directly. Frameworks are worth adopting when you need their specific abstractions, such as retrieval pipelines or complex chains, and they cost you when you do not, because they hide the wire format you need to debug malformed tool calls. Start small and adopt one when something concrete demands it.

How do I stop an agent from looping forever?

Two guards. A turn limit caps the number of round trips and returns a clear message when reached, rather than running until something else fails. A budget checks accumulated cost before each API call and stops the run if it would exceed the limit. Together they are the difference between a bug costing pennies and a bug costing hundreds of dollars overnight, and both are missing from most example agents.

Should I use the official SDKs or raw HTTP?

Raw HTTP is a defensible choice for a small agent. The SDKs are good but they churn, they pull dependency trees, and they hide the wire format — which is the thing you most need to see when a tool call comes back wrong. The HTTP APIs themselves are stable, so code written against them tends to still work months later without maintenance.

A working version of all of this

The Agent Starter Kit is this article as running code — Claude and OpenAI behind one interface, tool schemas generated from your Python type hints, cost tracking from the usage block including cache rates, a hard budget, and retries that honour retry-after. One dependency, no framework, and every example was executed against the live APIs before release with the token counts published.