How to Choose an LLM for Your AI Agent: Claude, GPT and Model Tiers

The best LLM for an AI agent is the cheapest model that finishes your own work items correctly, every time, inside your latency budget. It is not the one with the best leaderboard score or the most famous name. Claude, GPT and the other major families all handle tool calls well enough to run a business agent, so the choice comes down to six things you can measure: tool-use reliability, context needs, speed, cost per finished task, data terms and how easily you can switch later.

This guide is for owners and operations leads commissioning an agent, or wondering whether theirs runs on the right model. Model names change every few months, so we stick to durable criteria and say when we checked any name.

What should decide which model your agent uses?

Start from the job, not the brand. An agent that drafts refund replies has different needs from one that reads forty-page contracts or answers phones. Write down these six things for your job, in this order, before you look at any model list:

1. Tool-use reliability. Does the model pick the right tool, fill in the arguments correctly, and stop to ask when something is missing?
2. Context size. How much material must it hold in view at once: one email, or a whole case file?
3. Latency. Is a person waiting on screen, or does the work run in the background overnight?
4. Cost per task. Not cost per million tokens, but cost per finished work item, including retries.
5. Data terms. What does the vendor do with your inputs, how long are logs kept, and where is data stored?
6. Exit cost. If you must change model next year, how much of your system breaks?

If you are still deciding whether you need an agent at all, read what an AI agent is compared with a chatbot or plain automation first. Everything below assumes the answer is yes.

How much does tool-use reliability matter, and how do you check it?

It matters most. An agent is a model that decides which tool to call, so a wrong tool or an invented argument causes the failures people notice: a refund on the wrong order, a CRM record overwritten, a reply quoting a policy the model made up.

Anthropic's tool-use documentation describes a difference worth knowing about. When a required parameter is missing from the request, it says the larger Claude Opus models are much more likely to notice and ask for it, while Sonnet might ask but could also infer a plausible value, and that this is not guaranteed for less capable models. In plain terms, smaller and faster models tend to be more willing to guess. For a read-only lookup a guess is cheap. For a payment, it is not.

Two mechanisms help whichever family you pick. Both Anthropic and OpenAI offer a strict mode for tool definitions (strict: true) that makes the model's arguments follow your schema instead of a best effort. OpenAI's function-calling guide also advises keeping the number of tools available at the start of a turn small, with fewer than 20 as a soft target, and writing clear descriptions of when to use each one. Before comparing models, fix your tool design, because a vague tool description makes every model look unreliable.

To measure reliability, build a bench of twenty to fifty real work items from your last month, including ugly ones (missing order number, two customers with the same name). Run each candidate over them and count three things: the right tool, correct arguments, and a sensible stop when information is missing. Our guide to testing and evaluating AI agents covers how to score the results.

Small and fast, or large and careful: how do the tiers compare?

Every major vendor sells a ladder. The bottom rung is cheap and quick, the top rung reasons better on hard problems and costs several times more. The useful question is not which rung is best but which rung each step of your agent needs.

Anthropic's model-selection guidance suggests two ways to begin: start with the small model and move up only where tests show a gap, or start with the strongest model and move down once your tests pass. Both are sound. Pick the first when volume is high and the work is routine, the second when mistakes are expensive.

Anthropic also notes that its effort setting, which trades reasoning depth for speed and cost inside one model, is often a better lever than switching models. Try it before paying for a bigger tier.

Which current models fall on each rung?

As of November 2026, the two vendors most of our clients ask about list these lineups on their official models pages. Treat the names as a snapshot, because both vendors rename and replace models regularly.

Last verified November 2026 against Anthropic's models overview and OpenAI's models page. Prices, context windows and names change; check both pages before you budget.

Two numbers on those pages matter. Both vendors list context windows of roughly one million tokens, so raw size rarely decides the choice. And the price gap between rungs is large: Anthropic lists $0.10 to $10 per million input tokens from its smallest to its largest model. A task that works on the small rung costs a small fraction of the same task on the top rung.

Do you need a long context window?

Less often than the marketing suggests. A million tokens is roughly 555,000 words on Anthropic's current tokenizer, and most business agents work with one ticket, one invoice or one email thread plus a few retrieved passages.

The real question is accuracy over length. Put a fact on page 80 of a long document of your own, ask for it, and see whether the model finds it and ignores a similar-looking fact elsewhere. Long inputs also cost more, so check each vendor's pricing page for long-context rates. For a large knowledge base, retrieving the few relevant passages is usually cheaper and more accurate than pasting everything in, as we explain in custom GPT vs RAG chatbot vs AI agent.

What does a model really cost per finished task?

Price per million tokens misleads because agents loop. One work item might involve a plan, three tool calls, a retry and a final answer, each re-reading the growing conversation. Estimate cost per task like this:

1. Run twenty real items and record the input and output tokens from the API usage data.
2. Average them and multiply by the vendor's per-token prices.
3. Add the cost of failures: items a person must redo, and retries.
4. Compare cost per correct task across models, not per attempt.

The result often surprises people. A cheaper model that fails one task in five can cost more than a dearer one that rarely fails, once you count human rework. Three levers cut spend without changing models: prompt caching (Anthropic lists cache reads at 2.5 to 10 percent of the base input price depending on model), batch processing for non-urgent work (listed at 50 percent off), and routing, where a small model handles easy items and only hard ones go up a rung. Our companion piece on what it costs to run an AI agent goes through the budget in more detail.

What should you check in the vendor's data terms?

This is general information, not legal advice; terms differ by plan and region, so confirm with your lawyer. Check these five points on each vendor's official page:

• Training. Anthropic's privacy centre states that by default it does not use inputs or outputs from its commercial products to train its models, with an exception for feedback you explicitly submit. OpenAI's data-controls guide states that data sent to its API is not used to train or improve its models by default unless you opt in.
• Retention. OpenAI documents abuse-monitoring logs kept for up to 30 days by default, with zero data retention available to approved customers. For Anthropic, read the current retention terms for your agreement rather than relying on third-party summaries, which disagree.
• Location. OpenAI documents selectable data-storage regions for projects, with conditions. Anthropic's models are also sold through major cloud platforms, which set their own regional terms.
• Consumer versus business plans. Personal chat apps often have different settings from the API. Do not let staff paste client data into a personal account because the API terms are good.
• What the agent sends. Terms cover what you send, so the better protection is to send less: mask names, card numbers and health details before a prompt is built. See our guide to AI, personalisation and privacy for the compliance side.

Last verified November 2026. Sources: Anthropic Privacy Center and OpenAI data controls.

How do you avoid being locked into one vendor?

Assume you will change models at least once. Vendors retire models on published schedules (Anthropic's overview lists retirement dates for each model, currently no sooner than September 2027 for the models above), and a better or cheaper option will appear before then. Three design habits keep the switch cheap:

• Keep the model behind one thin layer. Your code asks for "classify this message" or "draft this reply", and one module translates that into a vendor call. Do not scatter vendor-specific calls through the system.
• Expose tools through a standard. The open Model Context Protocol lets different AI applications connect to the same tools. A tool built once as an MCP server can usually be used from more than one vendor's models. See our MCP server development service.
• Own your test set. Your bench of real work items is the asset that makes a model change a one-day job. Without it, every switch is a leap of faith. Prompts, tool definitions and tests should live in your repository, not in a vendor's dashboard.

Do not chase portability so far that you give up useful features. Use the vendor's strengths, such as strict tool schemas or caching, behind your own thin layer, and accept a day or two of rework when you switch.

Claude or GPT for business: which should you choose?

Neither wins in general. Both families call tools competently, both offer a ladder of sizes, and both publish business data terms. Test one from each on your own items and let the numbers choose. The differences that tend to matter are about fit rather than raw intelligence: behaviour when information is missing, how well a long list of rules in the system prompt is followed, handling of your language and document formats, and how latency and price match your volume.

A quick starting rule for the shortlist:

• Routing, tagging and extraction at high volume: shortlist the small tier from two vendors.
• Customer replies, back-office lookups and decisions with approval: shortlist the mid-tier from two vendors, with a person approving anything irreversible, as covered in human-in-the-loop approvals for AI agents.
• Complex reasoning, coding agents and rare edge cases: shortlist the top tier, then see whether lowering the effort setting keeps quality.
• Mixed work: route by step, small model first, escalating the hard ones.

We scope this as part of agentic AI development, using the Claude and OpenAI APIs.

Frequently asked questions

What is the best LLM for AI agents?

There is no single best one. The best model is the least expensive one that completes your own work items correctly within your speed limit. For most business agents that means a mid-tier model, with a small model for routing steps and a large model reserved for hard cases.

Is Claude or GPT better for business automation?

Both are capable of running business agents. Test one model from each vendor on twenty to fifty of your real work items and compare cost per correct task, handling of missing information, and data terms. The winner on your items beats the winner on a public benchmark.

Which AI model should I use for automation that runs in the background?

Background work tolerates latency, so look at price and accuracy first. Batch processing, which Anthropic lists at half price for non-urgent requests, can cut cost further. Start with the small tier and move up only where tests fail.

Do I need the newest model?

Not always. A newer model helps only if it passes your tests at a better price or with fewer failures. Pin the version in production, retest before upgrading, and read the vendor's retirement schedule.

Can I switch models later without rebuilding the agent?

Usually yes if you planned for it: a thin model layer, tools exposed through a standard such as MCP, and your own test set. Without those, expect real rework, because prompts tuned for one model often need adjusting for another.