FAQ
Frequently Asked Questions
Common questions about LLM API pricing, tokens, caching, and cost optimization.
Data built Nov 16, 2026. Verify with providers before decisions.
What is a token in LLM API pricing?
A token is a unit of text that LLMs process. Roughly, 1 token is about 4 characters or 0.75 words in English. Input tokens are the text you send to the model, and output tokens are the text the model generates back. Pricing is typically shown per 1 million tokens.
Why do output tokens cost more than input tokens?
Output generation requires the model to run inference step-by-step for each token, while input processing can be parallelized. On average across 2,076 chat models, output tokens cost 3.6x more than input tokens.
What is prompt caching and how does it save money?
Prompt caching stores previously processed input tokens so they don't need to be re-processed. Providers like OpenAI, Anthropic, and Google offer up to 90% discount on cached input tokens. This is especially valuable for multi-turn conversations or RAG workloads where the same context is repeated. See the prompt caching guide for details.
What is batch API pricing?
Batch APIs let you submit jobs asynchronously and typically offer 50% off standard pricing. The trade-off is higher latency (results are returned within hours instead of seconds). Good for offline processing, data enrichment, or non-interactive workloads. See the cache and batch pricing guide for more.
How do I estimate my monthly LLM API cost?
If you know your monthly input and output token totals, multiply each total by its per-token price and add the costs. Do not multiply monthly totals by request count again. If you have per-request token counts, first calculate the cost per request, then multiply by expected requests. Include cached input, retries, and billed tools where applicable. Use the calculator for workload estimates, or the compare tool to compare routes.
When should I self-host instead of using an API?
There is no universal monthly spend threshold. Compare the API bill for your workload with the GPU, setup, and operating costs of a self-hosted deployment that meets its throughput and latency requirements. Use the break-even calculator to explore the assumptions, or browse local models for open-weight options.
What is a context window and why does it matter?
The context window is the maximum number of tokens a model can process in a single request. Larger context windows allow more text, code, or data to be included, but input costs scale with context length. For example, a 128k-context request costs 4x more than a 32k-context request at the same per-token price. See the context window pricing guide for detailed comparisons.
What are reasoning models and why do they cost more?
Reasoning models (like OpenAI o3, Claude thinking, DeepSeek R1) use extended thinking to solve complex problems step-by-step. They cost 2-10x more than standard models because they generate many internal tokens before producing output. Use them for math, coding, and logic tasks where accuracy matters more than speed. See the reasoning pricing guide for details.
How do I compare models from different providers?
Use the compare tool to put 2-3 models side by side. You can compare pricing, context windows, modalities, and capabilities. Preset shortcuts help you find the cheapest chat, coding, or reasoning models across all providers. Browse the providers directory to see all available providers.
What is the model routing cascade?
Model routing cascade is a cost optimization strategy where you route simple queries to cheap models and complex queries to premium models. For example, a chatbot might route 80% of messages to a budget model and 20% to a premium model, cutting costs by 60-90% while maintaining quality for complex tasks. Read the model routing guide for a four-tier framework.
How much do embedding models cost?
Embedding models are much cheaper than chat models, typically $0.02-0.13 per million tokens. They convert text into numerical vectors for search and retrieval. Popular options include OpenAI text-embedding-3-small ($0.02/1M) and Cohere embed-v3 ($0.10/1M). See the embedding use case for cost examples.
What are open-weight models and local LLMs?
Open-weight models are models whose weights are publicly available, allowing you to run them on your own hardware. Popular families include Llama, Mistral, Qwen, and DeepSeek. Running them locally eliminates API costs but requires GPU hardware and engineering setup. See the local models page for available options and the local vs cloud comparison for cost analysis.
What is the cheapest LLM API for production use?
The lowest-cost suitable LLM API depends on the task, input and output token mix, caching eligibility, and provider route. First narrow models by required capabilities, context window, quality, and rate limits. Then compare current input and output rates for your expected volume in the calculator, including retries, billed tools, and eligible cache or batch pricing. A low list price alone does not prove the best production value.
How do I choose between GPT-4o, Claude Sonnet, and Gemini Flash?
GPT-4o, Claude Sonnet, and Gemini Flash are different model lines; available versions and provider routes can have different pricing, context limits, and capabilities. Select exact model IDs in Compare to check published prices and features, then test quality and latency with your own workload. Compare does not measure quality or latency.
How much does it cost to run an AI chatbot for 10,000 users?
If 10,000 users each send 10 messages per day, that is about 3 million chatbot messages in a 30-day month. The API cost depends on billed input and output tokens per message and the model's separate rates. A 500-token total alone is not enough to quote a monthly price without that split. Account for caching, retries, and extra model or tool calls, then use the calculator with explicit workload assumptions.
What is the difference between OpenAI API and Azure OpenAI?
OpenAI API provides direct access to OpenAI models, while Azure OpenAI provides access through Microsoft Azure. Supported versions, regions, deployment types, quotas, and billing can differ. Check the specific model and deployment you will use, and compare its current pricing. See the Azure vs OpenAI guide for details.
How do LLM API costs scale with usage?
LLM API costs scale with token usage, with input and output usually priced separately. Prompt caching can lower the price of cache-eligible input tokens, and batch pricing can lower eligible batch token rates when a model or provider publishes them. Your total savings depend on cache-hit share, output usage, batch availability, and workload mix. Use the calculator to model those assumptions for your workload, and see the cache and batch pricing guide for the pricing mechanics.
Have another question?
If you have a question not covered here, check the data sources page or use the calculator to estimate costs for your specific workload.