In plain English
An agent framework is the toolkit you use to build an LLM-powered agent — the loop that lets a model call tools, read the results, and keep going until a task is done. Picking one feels like a features contest: which has the nicest API, the most integrations, the best demo. That is the wrong question for anything you plan to ship.
Think of the difference between a go-kart and a car you drive on the highway. Both have a steering wheel and an engine, and on an empty parking lot they feel about the same. But the highway car needs seatbelts, brakes that work when wet, a fuel gauge, headlights, and a way to limp to the shoulder when something fails. A demo agent is a go-kart. A production agent carries real traffic, real money, and real users who will hit it in ways you never tested.
Production-readiness is the set of safety and operations features that separate the two: can you see what the agent did, can it recover when a tool times out, does it remember a half-finished task after a crash, and can you cap the cost before a runaway loop bills you for ten thousand model calls? This article is a checklist for judging any framework on those terms — not on how slick the hello-world looks.
Why it matters
The gap between "works in a notebook" and "works for 50,000 users" is where most agent projects quietly die. The model is rarely the problem. The problem is everything around the model — the parts a demo never exercises because a demo has one happy-path user, one network, and no money on the line.
Here is what actually breaks when an agent meets production:
- A tool times out and the whole run hangs or crashes, instead of retrying or failing gracefully. One slow API takes down every request behind it.
- A loop runs away. The agent decides to call the same search tool 400 times, each call billing the model API, and you find out from your invoice. Without a step cap, one bad prompt is a financial incident.
- Something goes wrong and you have no idea what. A user says "the agent gave me the wrong answer." Which tools did it call? What did the model see? Without tracing, you are debugging blind.
- The process restarts mid-task — a deploy, a crash, a scaled-down container — and the half-finished job is simply gone, because the agent's state lived only in memory.
- A high-stakes action fires with no human check. The agent issues a refund, deletes a record, or emails a customer, and there was no pause to approve it.
None of these show up in a feature comparison table. A framework can have 200 integrations and still leave you to build retries, tracing, and durable state by hand. The whole point of vetting for production-readiness is to find out before you commit whether the framework gives you these operational primitives — or whether you will be writing them yourself at 2am during an outage.
How it works: the five capability layers
Production-readiness is not one thing — it is a stack of operational layers, each of which a framework either gives you, lets you plug into, or leaves entirely to you. Picture them from the model outward: the closer a layer is to live traffic, the more it hurts when it is missing.
The demo only ever shows you the bottom layer — the agent loop. Everything above it is invisible until you put the agent in front of real users. When you evaluate a framework, you are really asking: for each layer, does this give it to me, expose a hook so I can add it, or force me to build it from scratch?
How a framework can support each layer
There are three levels of support, and the difference matters enormously for your timeline:
| Level | What it means | Your effort |
|---|---|---|
| Built in | The framework ships the capability — e.g. automatic retries with backoff, or a tracing dashboard. | Configure it. |
| Exposed via hooks | The framework gives you a clean extension point — callbacks, middleware, an event stream — so you can wire in your own tooling or a third-party service. | Integrate it. |
| Absent | No retry logic, no tracing hook, state lives only in a local variable. You patch around the framework. | Build it, and fight the framework. |
"Exposed via hooks" is often the sweet spot for serious systems: you do not want a framework's opinionated, hard-coded observability — you want it to emit clean events that flow into your monitoring stack. A framework with zero hooks for a layer is the real red flag, because patching around a framework that wasn't built for it is slow and fragile.
The production-readiness checklist
Here is the concrete checklist. Run a candidate framework through every item. The goal is not a perfect score — it is to know exactly which gaps you are signing up to fill yourself.
1. Observability — can you see what the agent did?
- Does it emit traces of every step: each model call, each tool call, the inputs and outputs of both? This is non-negotiable; you cannot debug an agent you cannot see.
- Can you attach structured logging and a trace/span ID that follows one request end to end?
- Does it integrate with standard tooling (OpenTelemetry, or a dedicated LLM-observability platform), or at least expose callbacks so you can?
- Are there evaluation hooks — a way to capture inputs and outputs to score quality offline later?
2. Reliability — does it survive failure?
- Retries with backoff on transient errors (rate limits, 503s, network blips), configurable per tool — not all-or-nothing.
- Timeouts on every tool call and on the overall run, so one hung dependency cannot hang the request.
- Idempotency support: if a step is retried, will it double-charge a card or send two emails? You need a way to make actions safe to repeat.
- Graceful degradation — when a tool is down, can the agent fall back, skip it, or return a partial answer instead of a hard crash?
3. State & durability — does it remember?
- Can a run's state be persisted to a real store (a database, not just memory) and resumed after a process restart or crash?
- For long or multi-turn tasks, can you checkpoint progress so a failure mid-way doesn't restart from zero?
- Is conversation/agent memory pluggable into your own storage, or locked to an in-process default that vanishes on redeploy?
4. Cost & rate control — can you cap the damage?
- A hard step/iteration cap on the agent loop, so a confused agent cannot call tools forever.
- Token or cost budgets per run, with a clean stop when exceeded.
- Visibility into token usage per step, so you can find the expensive parts.
- Sensible handling of provider rate limits (queueing or backoff) instead of hammering and failing.
5. Control & safety — is a human in the loop where it counts?
- Human-in-the-loop approval gates: can you pause the run before a high-stakes action (refund, delete, send) and require a person to approve?
- Streaming of partial output to the user, so a 30-second task doesn't look frozen.
- Integration points for guardrails — input/output filtering, prompt-injection defenses on anything the agent retrieves or a tool returns.
- Clear tool permission boundaries, so the agent can only reach the tools you intend.
A scorecard you can apply
Turn the checklist into a one-page scorecard. Rate each layer Built in (2) / Hooks (1) / Absent (0) for your shortlisted frameworks, then weight the rows by how much they matter for your product. A consumer chat agent weights human-in-the-loop lightly; an agent that moves money weights it as a hard requirement.
| Capability | Why it matters most | If absent, you must… |
|---|---|---|
| Tracing / observability | Debugging blind is impossible at scale | Wrap every tool + model call yourself |
| Retries & timeouts | Transient failures are constant | Add backoff and timeout logic everywhere |
| Durable, resumable state | Deploys and crashes happen daily | Build checkpointing into a database by hand |
| Step cap & cost budget | One bad loop = a real bill | Enforce limits in your own wrapper loop |
| Human-in-the-loop | Risky actions need a person | Build an approval queue and pause/resume |
| Streaming | Long tasks feel broken without it | Stitch streaming on top, fighting the API |
A practical rule: any layer scoring 0 (absent) for a capability your product genuinely needs is not a minor inconvenience — it is a separate sub-project you are committing to build and maintain. Two or three zeros in must-have rows is a strong signal to look at a different framework, or to accept that you are really building a platform, not using one.
Common pitfalls when vetting for production
- Judging by the demo. The README's 12-line example exercises the agent loop and nothing else. Build a small spike that adds a flaky tool, a crash mid-run, and a cost cap — that tells you the truth.
- Counting integrations instead of operations. "200 connectors" is a marketing number. One reliable tracing hook is worth more than 200 connectors you'll never use.
- Assuming memory is durable. Many frameworks default to in-process memory that looks persistent in a notebook and silently vanishes on redeploy. Confirm it can write to a real store.
- Ignoring idempotency until it bills a customer twice. Retries are only safe if repeated actions are safe. Check this before you turn retries on, not after.
- Letting the framework own your observability. A built-in dashboard you can't export from becomes a wall. Prefer frameworks that emit clean events into your stack over ones that trap data in their own UI.
- No step cap in development. It's tempting to skip the loop limit while building. Then a prompt change creates an infinite tool loop overnight. Set the cap on day one.
Notice that almost every pitfall is the same mistake in different clothing: evaluating the framework on its best day instead of its worst. The fix is always to deliberately reproduce a bad day in a spike before you commit.
Going deeper
Once the checklist is second nature, a few deeper themes separate a framework that merely survives production from one that makes operations pleasant.
Durable execution is the high bar. The strongest production setups treat an agent run like a workflow that can be killed and resumed at any point — every step checkpointed, side effects made idempotent, so a crash resumes exactly where it left off rather than re-running from the top. Frameworks built on a durable-execution engine give you this for free; with others you bolt it on. If your agents run for minutes or coordinate many tools, ask specifically how a run resumes after the process dies.
Provider SDKs versus orchestration frameworks. Lighter, provider-native toolkits like the Claude Agent SDK, the OpenAI Agents SDK, and Google's ADK tend to be closer to the model and lean on you for the outer operational layers. Heavier orchestration frameworks like LangGraph lead with durable state and human-in-the-loop as first-class features. Neither is better in the abstract — match the framework's built-in layers to the layers you do not want to own.
Multi-agent multiplies every requirement. The moment one agent calls another, your tracing must span agents, your cost cap must be global not per-agent, and a failure in a sub-agent must propagate sensibly. If a multi-agent design is on your roadmap, score the framework on whether observability and limits work across agents, not just within one.
Evaluation closes the loop. Production-readiness is not only about not crashing — it's about catching quality regressions. The frameworks that age best make it easy to capture real inputs and outputs and replay them against an eval suite, so you can prove a prompt or model change didn't make things worse before it reaches users.
The honest summary: the framework you pick decides which of these layers you configure versus which you build. There is no framework that gives you everything for free, and the right answer depends entirely on your product's risk profile. Score the layers, weight them for your use case, and choose with eyes open — see the full framework comparison to map specific tools onto these layers.
FAQ
What makes an agent framework production-ready?
A framework is production-ready when it gives you (or cleanly lets you add) the operational layers a demo never needs: tracing and logging so you can debug, retries and timeouts so transient failures don't crash you, durable state so a run can resume after a restart, step caps and cost budgets so a runaway loop can't bankrupt you, and human-in-the-loop gates for risky actions. Features are about a good day; production-readiness is about every bad day.
How do I evaluate an agent framework before committing to it?
Don't judge by the README demo, which only exercises the happy path. Build a small spike that adds a flaky tool, kills the process mid-run, and sets a cost cap, then watch how the framework behaves. Score it on observability, reliability, durable state, cost control, and human-in-the-loop — built-in, exposed via a hook, or absent — and weight those rows by what your product actually needs.
Why does an agent need retries and timeouts?
In production, tools fail constantly — rate limits, 503s, slow networks. Without per-tool timeouts, one hung dependency can freeze every request behind it; without retries with backoff, a transient blip becomes a user-visible failure. The catch is idempotency: a retried step must be safe to repeat, or you'll double-charge a card or send two emails, so check that before turning retries on.
Do I need durable state for an agent, or is memory enough?
For anything beyond a single short call, you need durable state. In-process memory looks fine in a notebook but vanishes on a crash, deploy, or container scale-down, taking the half-finished task with it. A production framework can persist a run's state to a real store and resume it where it left off. Confirm this explicitly — many frameworks default to in-process memory that silently disappears.
How do I stop an AI agent from running up huge costs?
Set a hard step or iteration cap on the agent loop so a confused agent can't call tools forever, add a token or cost budget per run that cleanly stops when exceeded, and track token usage per step so you can find expensive parts. Set the step cap on day one of development — most runaway-cost incidents come from skipping it while building and then shipping a prompt change that loops.
Should I use a framework at all for a production agent?
Not necessarily. Some teams go framework-free in production to own every operational layer — retries, state, cost control — at the cost of writing more glue code. The same checklist applies: you just fill every row yourself, deliberately. Use a framework when its built-in layers cover the ones you don't want to build; build from scratch when you need total control or your needs don't match any framework's opinions.