AI agent vs chatbot: what's the difference

A chatbot is a conversational interface over a body of knowledge. You send a message, it sends one back. An agent is a control loop: it decides what to do, does it using tools it has been given, looks at what happened, and decides again. The difference is not conversational quality. It is whether the system can change the state of your business.

A well-built chatbot is a genuinely useful thing and often the right answer. Point it at your documentation, your policies, and your product help centre, and it will handle the large share of questions that are really lookups: what your refund window is, whether a feature exists, how to reset a password.

It stops being enough at the point where the honest reply is "I can tell you the policy, but I cannot do the thing." A customer who asks to change their delivery address does not want the policy on address changes. Every one of those conversations ends in a handoff, and the chatbot has added a step rather than removing one.

Three capabilities turn a model call into an agent, and each one adds engineering work rather than prompt text.

1. Planning. The model decides the sequence of steps at runtime instead of following a fixed script. Interleaving reasoning with actions in this way is the pattern described in the ReAct paper, and it is why an agent can handle requests nobody anticipated.
2. Tool calling. The model is given a set of functions with typed inputs and can choose to invoke them. This is the mechanism that lets it read and write your systems, and the reason an agent needs a permissions model that a chatbot does not.
3. State. The agent tracks what it has already done within a task, so a multi-step job survives a retry without repeating a side effect. This is the part teams most often underestimate.

Note what is not on that list: a bigger model. Moving from a chatbot to an agent is an architectural change, not a model upgrade. Anthropic's guidance on building effective agents makes the same point from the other direction, recommending the simplest pattern that solves the problem and reaching for a full agent loop only when the task genuinely requires one.

Take one request: "I was charged twice for last month, can you fix it?"

The chatbot retrieves your billing policy and explains that duplicate charges are refunded within five working days once verified, then suggests contacting support. Accurate, and the customer still has an unresolved problem and a second message to write.

The agent looks up the account, queries the payment provider for charges in the period, finds two charges with the same amount and near-identical timestamps, checks the refund against the policy limit it is allowed to act within, issues the refund, writes a note on the ticket, and replies with the amount and the expected arrival date. If the amount had exceeded its limit, or if the two charges turned out to be a subscription plus a one-off, it stops and hands the case to a person with its findings attached.

The second version is more useful and considerably more work. It needs read access to accounts, read access to the payment provider, scoped write access to refunds, a written rule about what it may decide alone, and a path for the cases it must not decide.

A chatbot's worst day is a wrong answer. An agent's worst day is a wrong action performed competently: the refund issued twice, the wrong record updated, the email sent to the whole list. The model's confidence is identical in both the correct and the incorrect case, so confidence cannot be your safety mechanism.

What works instead is unglamorous and mostly predates language models. Give each tool the narrowest permissions that let it do its job. Make write operations idempotent so a retry cannot double-charge anyone. Put a spending or blast-radius limit on anything irreversible. Log every tool call with its inputs and its result so a bad outcome can be reconstructed rather than guessed at. Run an evaluation set of whole tasks, scored on the end state, before every change reaches production.

Teams often treat escalation as the thing that happens when the agent fails. Built that way, handoff is a dead end: the customer repeats themselves and the human starts from nothing.

Treat it as a first-class path instead. Decide up front which categories of decision the agent may never make, and make it stop on those by design rather than by luck. When it stops, pass on what it already established: the account, what it checked, what it found, and what it believes the right action is. A good handoff turns a fifteen-minute investigation into a one-minute approval, which is where much of the real return sits.

1. Does resolving the request require changing something in another system? If no, build a chatbot. Most of the value in a knowledge-heavy support queue is reachable without an agent at all.
2. Is the sequence of steps the same every time? If yes, you may not need a model in the loop for the orchestration. A deterministic workflow with a model at the one genuinely ambiguous step is cheaper to build and far easier to debug.
3. What is the worst thing a wrong action could do? If the answer involves money, contracts, or data you cannot restore, you need approval gates and audit logging from the first day, and that belongs in the initial scope rather than a later hardening phase.

Start with retrieval over your own content and ship it. It is faster to build, it tells you what people actually ask, and the retrieval layer is reused unchanged when you later add tools. Then pick the single highest-volume request that currently ends in a handoff and give the system exactly the tools needed to resolve that one, with a hard limit on what it may decide alone.

That sequencing gets you a working system early and an agent whose scope is set by evidence rather than ambition. It also means the integrations you build first are the ones you already know get used.