Agentic AI token consumption and cost estimation guide
Multi-step agentic workflows consume tokens very differently from single-turn calls: context accumulates across steps, tool definitions and results add tokens, and reasoning can add billed output tokens. This guide explains the patterns and provides an illustrative cost-estimation method.
Why agents cost more than single-turn queries
A single-turn LLM query sends a prompt and receives a response. An agentic loop does this repeatedly: each step sends the full accumulated conversation history, new tool results, and task instructions. Repeated calls can cost more than a comparable single-turn response, depending on call count, context reuse, and model pricing.
Three key cost drivers differentiate agentic from single-turn workloads:
- Accumulating context window — every step re-sends prior messages, tool results, and system instructions
- Tool-call token overhead — tool schemas, arguments, and results can add input and output tokens; measure them per workload
- Reasoning tokens — additional billed output may vary with the model and task; account for actual usage
Agent-suitable models in the database
Of 3092 total models in the catalog, 295 support both
responseSchema (structured output / tool calling) and systemMessages with valid pricing —
making them suitable for agentic workflows. 189 of these also support
reasoning/chain-of-thought for planning and analysis.
Total cost per million tokens ranges from $$0.060 (cheapest non-reasoning, deepseek-ocr) to $$0.190 (cheapest reasoning, gpt-oss-20b) — a 3x spread.
Compare cheapest agent models →Agent token consumption patterns
Pattern A — Simple chain
Step 1 → Step 2 → Step 3. Context grows linearly. Total tokens = sum of (accumulated context + new instructions + output) per step. Each step re-sends the full prior conversation.
Example: a three-call code-review agent sends 2,000 input tokens in step 1, 4,000 in step 2, and 6,000 in step 3, with 1,000 output tokens per call. The run uses 12,000 input and 3,000 output tokens in total. Later inputs include re-sent history, not just new instructions.
When using API usage records, add each call’s reported input and output totals once. In OpenAI usage reports, cached input is already part of input tokens, while reasoning and generated tool-call arguments are part of output tokens. Apply the relevant cached-input price rather than adding cached tokens again. Count retries, subagent calls, and separately billed hosted tools separately. Other providers may report usage differently.
Pattern B — Supervisor + workers
Planner delegates sub-tasks (fan-out, merge results). A reasoning planner decomposes the task into parallel subtasks that cheap executors handle. The merge step re-sends all worker outputs, making the final step the most expensive.
Pattern C — Loop with reflection
Agent acts, observes, reflects, retries. Context grows fastest. Each retry re-sends the entire history plus the new attempt. This pattern dominates debugging and iterative code-generation agents.
Cost scenarios
These are illustrative token-only estimates based on the stated request volumes and catalog rates, not measured workload savings. Cache-write fees, storage/TTL, external tools, retries, and latency are excluded; evaluate task accuracy separately.
Each monthly cost is the arithmetic average of the listed models' individual estimates: it assumes the stated runs are split equally among those models. The cascade row is an illustrative routing mix, not the cost of calling every listed model for every run. Real escalation rates, token counts, and success rates require a workload-specific estimate.
Simple code-review agent (3 steps)
Input: 12.0K · Output: 3.0K · Volume: 50,000 runs/month
| Strategy | Models | Monthly cost | vs All-Premium |
|---|---|---|---|
| Budget only | deepseek-ocr, l3-8b-lunaris, qwen2.5-7b-instruct | $37.50 | -50% |
| Cascade | deepseek-ocr, gpt-oss-20b | $34.50 | -54% |
| Premium only | gpt-oss-20b, gpt-oss-120b, deepseek-r1-distill-qwen-14b | $75.50 | — |
Web research agent (5 steps)
Input: 25.0K · Output: 4.0K · Volume: 10,000 runs/month
| Strategy | Models | Monthly cost | vs All-Premium |
|---|---|---|---|
| Budget only | deepseek-ocr, l3-8b-lunaris, qwen2.5-7b-instruct | $14.50 | -47% |
| Cascade | deepseek-ocr, gpt-oss-20b | $12.35 | -55% |
| Premium only | gpt-oss-20b, gpt-oss-120b, deepseek-r1-distill-qwen-14b | $27.33 | — |
Customer support triage (3-7 steps)
Input: 12.0K · Output: 1.5K · Volume: 100,000 runs/month
| Strategy | Models | Monthly cost | vs All-Premium |
|---|---|---|---|
| Budget only | deepseek-ocr, l3-8b-lunaris, qwen2.5-7b-instruct | $67.50 | -91% |
| Cascade | deepseek-ocr, qwen3-coder-30b-a3b-instruct | $82.50 | -89% |
| Premium only | gpt-4.1-mini, gpt-4.1-mini-2025-04-14, GPT-4.1 mini | $720.00 | — |
Compare cost per successful task
Token rates show what calls cost, but not what it costs to deliver an accepted result. For the same workload and measurement period, divide total billable API spend, including retries and agent steps, by the number of successfully completed tasks. Failed attempts still count toward spend.
Hypothetical example: route A attempts 100 tasks, spends $40, and produces 40 accepted results ($1.00 per successful task). Route B attempts the same 100 tasks, spends $60, and produces 80 accepted results ($0.75 per successful task). Route B costs more in total but less per accepted result in this example. These are invented numbers, not measured model-quality claims.
Use the Calculator to estimate token charges, then measure actual billable usage and outcomes using the same acceptance criteria for both routes. General benchmark scores cannot establish a workflow's success rate. Dividing cost per attempt by an assumed success rate can miss retries and additional tool charges.
Optimization strategies
Prompt caching
Keep reusable system instructions and tool definitions at a stable prompt prefix where the provider supports caching.
The catalog field cachedInputUsdPer1MTokens is a model/route-specific cached-input rate, not a universal discount.
Cache-write charges, time-to-live, and storage costs vary by provider and model; check the applicable
OpenAI,
Anthropic, and
Gemini terms.
In the illustrated web-research scenario, reusing the system prompt and tool definitions may lower repeated input charges. Actual savings depend on cache hits, write and retention fees, and the tokens reused.
Model cascade routing
Consider a lower-cost model for classification, then use a reasoning model for analysis or verification when needed. The cost difference depends on routing, output volume, retries, and task accuracy; compare the scenarios rather than assuming a fixed saving.
Context window budgeting
Shorter history is not always cheaper: rewriting a shared prompt prefix can lose cached-input discounts, and agents may re-fetch discarded tool results. Compare compaction with an unchanged-prefix baseline using billed cache-read and cache-write tokens, extra calls, and cost per successful task. A 2026 coding-agent study observed this trade-off; OpenAI's prefix guidance explains why cache reuse can fall after a rewrite.
Parallel sub-tasks
Fan out to cheap executor models and merge results with one reasoning call. This keeps context growth flat for the executor branches and concentrates cost on the merge step.
Key takeaways
- The gap between cheapest agent-suitable ($$0.060/M total) and premium reasoning ($$2.00/M+ total) is 33x — this is a token-rate comparison, not a measure of task quality.
- A cascade can reduce estimated token charges for some workloads; validate accuracy and retries against an all-reasoning baseline.
- Tool schemas, calls, results, and retries consume tokens beyond the visible conversation; measure actual usage.
- Prompt caching can help recurring agent calls when cache hits are reliable and model-specific write/storage charges are worthwhile.
Pricing data sourced from our catalog. Check data sources for provenance and freshness.