How to Think About AI Agents

Where I Started

On having a mental model before you have an architecture.

"Agent" is doing a lot of work as a word right now. It covers everything from a single function call wrapped around a model to elaborate multi-agent systems with planners, critics, and memory stores. That ambiguity is where a lot of overbuilt systems come from: people reach for the most complex pattern before they've decided what an agent even is for their problem.

The mental model I keep coming back to is to treat an agent like a capable but green teammate — one with near-perfect recall and zero judgment about when to stop, ask, or doubt itself. That framing is useful because it tells you exactly where the engineering goes: not into making the model smarter, but into giving it good tools, clear instructions about when to use them, and guardrails for the moments its confidence outruns its competence.

What's Actually Under the Hood

Strip away the vocabulary and an agent is a loop: the model decides to call a tool, you execute it, you feed the result back, and it decides again — until it's done. Most of what makes that loop work (or fail) is unglamorous:

• Tool definitions. A good tool description says clearly when to use it and what arguments it takes. Ambiguous tools produce ambiguous behavior; this is the highest-leverage thing you can fix.
• The execution loop. Registering tools, running them, handling errors, and deciding when to stop. The control flow is yours, not the model's.
• Protocols like MCP. The Model Context Protocol is making tool integration something you can standardize rather than hand-roll per app — worth understanding even if you don't adopt it yet.
• Architecture, chosen deliberately. Single-agent-with-tools handles more than people expect. Multi-agent setups buy you specialization at the cost of coordination complexity and latency. Reach for them when you can name the problem they solve.

The Most Common Mistake

It's over-engineering. A planner, three specialized sub-agents, and a shared memory bus — for a task a single well-prompted model with two tools would have nailed. Complexity is a cost you pay on every request, in latency, in tokens, and in debuggability. The best agentic system is usually the simplest one that meets the bar.

Know the mental model first. The architecture follows from it — and so does the discipline to not build one when a plain call will do.

This is a companion to Chapter 8: Agentic Systems in my book, AI Engineering: Building Production-Ready LLM Applications. The full chapter covers tool-use mechanics, the Model Context Protocol, agent architectures in practice, and production case studies — with runnable code.

Read the full chapter free → book.jameshu.io