Prompt Caching Explained (2026): 0.05x Reads, 5-Minute TTLs, Real Math

The first time I added prompt caching to an agent loop, the bill dropped and nothing else changed, which made me suspicious enough to read every pricing footnote. Prompt caching is the closest thing the LLM APIs have to a free lunch: the provider keeps the attention state it already computed for the start of your prompt, and charges you a fraction of the input price to reuse it. It is also easy to get wrong silently, because a cache miss looks exactly like a cache hit except on the invoice. This post explains how prompt caching works under the hood, how Anthropic, OpenAI and Google implement it in late 2026, what the TTLs and multipliers actually are, how it stacks with batch APIs, and a worked example of the arithmetic.

What is prompt caching, and what does it actually store?

Every request to a transformer model has two phases. Prefill reads your whole prompt and computes, for every token and every layer, a key and a value vector that later tokens attend to. Decode then generates the output one token at a time, reusing those vectors. The key/value tensors are the KV cache. Prefill is where input tokens are spent, and for a long system prompt plus tools plus conversation history it is most of the work.

Prompt caching keeps those KV tensors after the request ends and reuses them when the next request starts with the same tokens. The open-source engines show the mechanism plainly. In vLLM's automatic prefix caching design, the KV cache is split into fixed-size blocks, each block is identified by a hash of its parent block's hash plus its own tokens (plus extras such as LoRA IDs and tenant salts), only full blocks are cached, and freed blocks are evicted least-recently-used first. SGLang's RadixAttention paper does the same job with a radix tree over token sequences. I covered where this sits in a serving stack in LLM inference in 2026.

The parent-hash chaining is the one fact that explains every caching rule you will meet: a block is only reusable if everything before it is byte-identical. Change one token at position 200 and every block after position 200 is a different hash. That is why it is a prefix cache, not a "similar prompt" cache, and why a timestamp at the top of a system prompt quietly disables it.

Hosted APIs store derived state, not your text. Anthropic's prompt caching docs say KV representations and hashes are held in memory only, not stored at rest, and that caches are isolated between organisations and, on its own API, between workspaces. The model also does not treat cached tokens differently: Google's explicit caching page states that cached content is simply a prefix to the prompt. Same input, same output distribution, lower price.

How Anthropic, OpenAI and Gemini implement prompt caching in 2026

The three big APIs converged on similar economics and very different ergonomics. Here is the state as of today, from each vendor's own docs and pricing pages.

	Anthropic (Claude)	OpenAI (GPT-5.6 and later)	OpenAI (earlier models)	Google (Gemini)
How you turn it on	cache_control on blocks, or one top-level cache_control for automatic placement	Implicit by default; explicit prompt_cache_breakpoint on content blocks	Automatic; prompt_cache_key steers routing	Implicit by default on 2.5 and newer; explicit caches.create on Gemini 3 and 2.5 (beta)
Minimum prefix	512 tokens on the newest models, up to 4,096 on Opus 4.6 and Haiku 4.5	1,024 visible input tokens	Varies by request settings	4,096 (Gemini 3.x, 4 Argon), 2,048 (2.5) for implicit
Write cost	1.25x input (5 min), 2x (1 hour)	1.25x (30 min), 2.5x (12 h)	No write charge	Implicit: none. Explicit: cache price plus storage per hour
Read cost	0.1x; 0.05x on Opus 5.5 and Sonnet 5.5; 0.025x on Fable 5.1	0.1x; 0.05x on GPT-6.1 Sol and Terra	Model-dependent cached-input rate	0.1x on 3.8 Flash and 3.1 Pro; 0.05x on 4 Argon
Lifetime	5 min or 1 h, refreshed on every hit	30 min default, 12 h option	in_memory (5 to 10 min idle, up to 1 h) or 24h; GPT-5.5 is 24h only	Explicit TTL you set, default 1 hour
Where hits show up	usage.cache_read_input_tokens	usage.input_tokens_details.cached_tokens	Same field	usage_metadata.cached_content_token_count

Sources: Anthropic pricing and caching docs, OpenAI prompt caching guide and pricing, Google's implicit caching, explicit caching and pricing pages.

A few things in that table deserve a second look.

Anthropic made reads cheaper on its newest models. Claude Opus 5.5 lists $4 per million input tokens and $0.20 for cache hits, and Sonnet 5.5 lists $2 and $0.10, both 0.05x. Fable 5.1 reads at $0.25 against a $10 base, 0.025x. Older models stay at 0.1x. When a read costs 2.5% of a fresh token, keeping a prefix warm is nearly free and missing it is comparatively expensive.

OpenAI started charging for writes. For models before GPT-5.6 the guide says there is no cache-write charge: caching was automatic. From GPT-5.6, writes cost 1.25x for the 30-minute lifetime and 2.5x for 12 hours, and you get explicit breakpoints (up to four per request) in exchange. Reads stay at 0.1x on most of these models and drop to 0.05x on GPT-6.1 Sol and Terra: the pricing page lists GPT-6.1 Terra at $1 input and $0.05 cached. The guide also notes that caches live on individual machines and traffic above roughly 15 requests per minute on one prefix can overflow to machines without it, which is what prompt_cache_key was for on older models.

Google split caching in two. Implicit caching is on by default for Gemini 2.5 and newer, with "no cost saving guarantee" in Google's words. Explicit caching guarantees the discount but bills storage for the whole TTL: $0.50 per million tokens per hour on Gemini 3.8 Flash through 31 December 2026, $4.50 on Gemini 3.1 Pro Preview. Explicit caching is not supported on Gemini 4 Argon and later, which are implicit only; Argon lists cached input at $0.10 against $2 input through 30 November 2026 (both double on 1 December), so its implicit hits are 0.05x.

Prompt caching TTLs: 5 minutes, 1 hour, or 24 hours?

The TTL question is about the gap between requests that share a prefix. On Anthropic, a cache read refreshes the timer at no extra cost, so a 5-minute entry stays alive indefinitely as long as some request touches it every five minutes. The 1-hour TTL pays for itself in the 5 to 60 minute band: a user who reads for ten minutes before replying, or an agent whose tool call takes longer than five minutes.

The break-even is simple arithmetic from the published multipliers. With a 5-minute write at 1.25x and reads at 0.1x, two requests cost 1.35x against 2x uncached, so one hit already pays. With the 1-hour write at 2x you need two reads to come out ahead, as Anthropic's pricing page states.

The lifetime counts from the start of the request, not the end. A generation that takes four minutes leaves about one minute for the next request on a 5-minute entry.

Designing prompts for cache hits

Because the cache is a prefix match, the design rule is ordering by stability: the things that never change go first, things that change per session next, things that change per request last. Anthropic renders tools, then system, then messages, so a breakpoint at the end of the system prompt caches the tool definitions too.

The common ways to break this are boring, which is why they pass code review:

• A date, request ID or user name interpolated into the system prompt.
• Tool definitions built from a dict or set with no stable order, or json.dumps without sorted keys.
• Per-user tool sets, so nothing is shared across users.
• Switching model mid-conversation. Caches are scoped to a model, so a router that bounces between two models keeps two cold caches.
• Fanning out parallel requests before the first one has written the cache. On Anthropic an entry is readable only once the first response starts streaming.

The placement subtlety is the shared-prefix, varying-suffix case. If 10,000 requests share a 30,000-token instruction block and differ in the final question, the breakpoint belongs at the end of the shared block. Put it at the end of the whole prompt and every request pays to write an entry nobody reads. The same logic applies to OpenAI's implicit mode on GPT-5.6, which places its breakpoint at the end of the latest eligible message; for that shape, explicit breakpoints are the cheaper choice now that writes cost 1.25x. This is the same discipline as context engineering: decide what is stable and lay the prompt out accordingly.

Two Anthropic-specific limits catch agent builders. Each breakpoint looks back at most 20 blocks for an earlier entry, so a single turn that appends a long run of blocks can lose its cache unless you add an intermediate breakpoint. And there is a maximum of four breakpoints per request.

How prompt caching stacks with batch APIs

All three vendors sell asynchronous batch processing at half price, and the discounts compose.

	Anthropic Message Batches	OpenAI Batch API	Gemini Batch API
Discount	50% on input and output	50%	50%
Turnaround	Most within 1 hour, expires at 24 h	24 h completion window	24 h target, usually faster
Size limit	100,000 requests or 256 MB	50,000 requests, 200 MB file	2 GB JSONL input file
Results kept	29 days	Output file deleted after 30 days	6 weeks by default
Caching inside batches	Multipliers stack; hits are best effort	Cached input listed at batch rates	Batch caching prices listed

Sources: Anthropic batch processing, OpenAI batch guide, Gemini Batch API.

Anthropic's pricing page states the caching multipliers stack with the batch discount. The catch is that batch requests run concurrently and in any order, so hits are best effort. Anthropic's own FAQ suggests sending one request with the shared prefix on a 1-hour TTL first, waiting for it, then submitting the rest. OpenAI's pricing table lists Flex processing at the same prices as Batch, which is the option for work that can tolerate slower responses but still wants a synchronous API shape.

The real savings arithmetic

Here is a worked example from published prices, not a production measurement. A support assistant on Claude Sonnet 5.5 sends 100,000 requests a month. Each has a 30,000-token shared prefix (instructions, policy docs, tool definitions), 1,000 tokens of unique input and 500 tokens of output. Sonnet 5.5 is $2 per million input, $2.50 for 5-minute writes, $0.10 for reads, $10 per million output.

• No caching: 3.1 billion input tokens at $2 is $6,200, plus 50 million output tokens at $10 is $500. Total $6,700.
• Cached, warm: 3.0 billion tokens read at $0.10 is $300, 100 million uncached at $2 is $200, output $500. If the cache goes cold once a day and is rewritten, that adds 30 writes of 30,000 tokens at $2.50, about $2. Total about $1,002, an 85% cut.
• Cached at a 70% hit rate: every miss is a write at 1.25x. Reads $210, writes 900 million tokens at $2.50 is $2,250, uncached $200, output $500. Total $3,160. Most of the saving is gone.
• Batch, no cache: half of $6,700, $3,350.
• Batch plus cache, best case: reads at $0.05, uncached at $1, output at $5: $150 + $100 + $250 = $500.

The 70% row is the one I would put on a wall. Caching is not a percentage discount you switch on; it is a hit rate you have to earn and then defend. Teams that track spend the way I described in FinOps for AI should be tracking hit rate per route as a first-class number, alongside tokens and dollars.

The other lesson is that output is untouched by caching. In the warm case, output is half of the remaining bill. That is where batch, model routing and output caps come in, and it is what the companion guide on how to cut LLM API costs with caching, batching and routing works through with code.

How to measure your cache hit rate

Every provider reports cached tokens in the response, and that is the only ground truth. On Anthropic the uncached remainder is input_tokens, and the full prompt is input_tokens + cache_creation_input_tokens + cache_read_input_tokens, so a hit rate is reads divided by that sum. A minimal check after any change to prompt assembly:

# Python 3.12, anthropic==2.3.0
# Hit rate for one response; aggregate these per route in your metrics.
def cache_hit_rate(usage) -> float:
    total = (usage.input_tokens
             + usage.cache_creation_input_tokens
             + usage.cache_read_input_tokens)
    return usage.cache_read_input_tokens / total if total else 0.0

# A healthy multi-turn loop: reads grow every turn, writes stay small
# (roughly the last turn's new content), input_tokens is just the tail.

On OpenAI, read usage.input_tokens_details.cached_tokens and, from GPT-5.6, cache_write_tokens. On Gemini's generateContent API it is usage_metadata.cached_content_token_count. Export them as metrics and alert when the ratio drops after a deploy. Platform discipline still matters here: the failure is a prompt-template change merged on a Friday, not a model problem.

FAQ

Is prompt caching the same as semantic caching?

No. Prompt caching reuses the model's KV state for an identical token prefix and still runs the model, so you get a fresh answer. Semantic caching, usually in a gateway, returns a stored answer for a similar question without calling the model at all.

Does prompt caching change the model's output?

It should not. The cached KV tensors are the same values prefill would compute again, and Google's docs state the model makes no distinction between cached and regular input tokens. What changes is price and time to first token.

How long does a prompt cache last?

On Anthropic, 5 minutes by default or 1 hour at a higher write price, refreshed by each hit. On OpenAI GPT-5.6 and later, 30 minutes by default with a 12-hour option; earlier models use in-memory retention or 24 hours. On Gemini explicit caching, whatever TTL you set, defaulting to 1 hour, with storage billed for the whole period.

Is prompt caching worth it for short prompts?

Often it does not apply at all. Prefixes under the minimum (512 to 4,096 tokens on Claude depending on model, 1,024 on GPT-5.6, 2,048 to 4,096 for Gemini implicit caching) are processed normally with no error. If you sit just under the threshold, padding the stable part with useful examples can be cheaper than not caching.

This quarter I would do three things: put cache hit rate per route on the same dashboard as token spend, move every timestamp and request ID out of system prompts, and push every job nobody waits for onto a batch API with a warmed 1-hour prefix. None of it needs a new vendor, which is the best kind of cost work.

Related: FinOps for AI in 2026, LLM Inference in 2026 and Self-Hosting LLMs vs API Costs in 2026.