What actually drives an LLM API bill
Tokens are not words, output costs several times more than input, and caching changes the arithmetic entirely. The three things that decide what you pay.
Updated 2026-08-03
API pricing looks simple — a rate per million tokens — and then the first invoice arrives at several times the estimate. Three things account for nearly all of the gap, and none of them is on the pricing page in a form you can act on. Tokens are not words and the ratio depends on your language. Output is priced several times higher than input. And caching, where it exists, changes the economics of a long prompt completely.
A token is not a word
Models read subword fragments produced by byte-pair encoding, and the ratio to anything human-countable depends heavily on what you are writing. English prose is the best case, because that is what the tokenizers were fitted to.
| Content | Roughly | Meaning |
|---|---|---|
| English prose | ~4 characters per token | About 750 words per 1,000 tokens |
| Code | ~3 characters per token | Punctuation and indentation each cost |
| JSON | ~3 characters per token | Braces, quotes and keys are all tokens |
| Chinese, Japanese, Korean | ~1 character per token | The same meaning costs several times more |
| Base64 or a hash | ~2 characters per token | Random strings tokenize terribly |
Output is the expensive side
Across essentially every provider, generated tokens cost three to five times what input tokens cost. The practical consequence runs against instinct: trimming your system prompt saves less than trimming the answer. A model asked for structured JSON rather than prose-with-an-explanation can cut the expensive half of the bill by more than half.
- Ask for the format you actually need. "Reply with JSON only, no commentary" is a cost control.
- Cap it. A `max_tokens` limit is a budget, not just a safety valve.
- Do not ask a model to restate the question before answering it. You are paying for both.
- Reasoning modes bill their thinking as output. That is often worth it — and it is worth knowing.
Caching is the biggest single lever
If your workload sends the same long system prompt or the same document across many calls, prompt caching discounts that repeated portion — commonly by 50% to 90%. It is not automatic everywhere, it requires the repeated part to be at the start of the prompt, and it usually has a minimum length. Where it applies, it changes what a long prompt is worth doing.
| Pattern | Cacheable? | Effect |
|---|---|---|
| A fixed system prompt across many calls | Yes | Large — this is the case caching was built for |
| One long document, many questions about it | Yes | Large, if the document leads the prompt |
| A different user message each time | No | None — it is different every call |
| Variable content placed before fixed content | No | None, and it is a common accident. Order matters |
Longer context is not free, and often not better
A million-token window is a capability, not an instruction. Cost scales with what you send on every call, and accuracy degrades in the middle of a long input — the well-documented "lost in the middle" effect, where models attend reliably to the beginning and end and less so to the middle. Retrieving the three relevant paragraphs generally beats sending the whole document, on both axes at once.
A worked example
A support-triage workload: 2,000 tokens of input and 500 of output per call, a thousand calls a day, at $3 and $15 per million.
| Change | Cost per day | Difference |
|---|---|---|
| Baseline | $13.50 | — |
| Ask for JSON instead of prose — output halves | $9.75 | −28% |
| Cache the 1,500-token system prompt at 90% | $5.70 | −58% |
| Both, on a tier priced 4× lower | $1.43 | −89% |
Questions
Why is my non-English workload so much more expensive?
Tokenizers are fitted predominantly to English text, so other scripts fragment into more tokens. A Japanese sentence can cost three to four times what its English translation costs — the same meaning, several times the tokens.
How accurate is a character-based token estimate?
Within about 10–15% for ordinary text, which is enough to check that a prompt fits and to size a budget. It is not enough to reconcile an invoice — for that, use the provider's reported usage, which is what they actually bill on.
Should I always use the cheapest model?
No, but you should test it. Classification, extraction and routing are frequently done just as well by a tier costing twenty times less, and those are usually the highest-volume calls in a system. Reserve the expensive model for the work that visibly needs it.
Does a longer system prompt make the model follow instructions better?
Up to a point, then it reverses. Four clear rules are followed more reliably than fifteen hedged ones, and the system prompt is billed on every single request — so length there is a recurring cost for a diminishing return.