Prompt caching is a bet, not a switch

Everyone will tell you the same thing when the AI bill climbs: turn on prompt caching. It is usually good advice. It is also the most misunderstood cost optimization in AI, because the way it is described — “cache your prompts to save money” — hides the one fact that decides whether it helps you at all.

Here is the shape of the misunderstanding. Two teams run the same kind of workload on the same model. Both enable prompt caching with the same one-line change. One team’s cost per request drops sharply. The other team’s bill barely moves — and for a short window, right after they turn it on, it actually goes up. Same feature, same model, opposite outcomes. Nobody flipped a wrong switch.

That happens because prompt caching is not a discount you switch on. It is a bet. You are wagering that a large, unchanging piece of your context will be reused enough times, soon enough, to earn back the cost of storing it. For some workloads that bet is nearly free money. For others it never pays off. Knowing which one you have — before you touch a dashboard — is the whole skill, and it is what this guide teaches.

The Hidden Economics of AI Coding guide makes the case that caching is the highest-leverage first move for repeated-context workloads, and notes that it “isn’t magic, just an engineering tool that pays off when the workload has reusable structure.” This guide is the mechanics behind that caveat: what actually gets cached, why a cache write can cost more than an ordinary token, and how to look at a workload and predict the sign and rough size of the savings.

What prompt caching actually caches

Prompt caching caches your input, not the model’s output. Most of the confusion around caching traces back to missing that, so start there.

When you send a request to a language model, the provider has to process every token of the prompt before it can generate a single token of a reply. For a coding agent, a retrieval system, or a long chat, most of those input tokens are the same on every call: the system prompt, the tool definitions, the retrieved documents, the conversation so far. Only the tail end changes — the new user turn, the latest tool result. Prompt caching lets the provider store the processed form of that repeated leading section and charge a reduced price to reuse it, instead of reprocessing it from scratch each time.

Two terms carry the rest of this guide, so pin them down now. “Prefix” here means exactly what it sounds like: the beginning of the request. The model reads your prompt in order, top to bottom, and the provider matches an incoming request against its cache the same way — starting at the very first token and stopping at the first token that differs. The stable prefix is the stretch from token one up to that first point of change: everything at the front of the request that is identical on every call. It is the only part eligible to be cached, which is why the technique cares about where content sits, not just whether it repeats — repeated content buried behind something that changes is not a prefix and earns nothing. The volatile tail is everything after that point, the part that changes from request to request. Caching operates on the prefix. The tail is always processed fresh, and the model’s generated output is never cached — it is produced new every time and billed at the normal output rate.

This is also why prompt caching is a different technique from semantic caching or response caching, which store a finished answer and return it when a similar question comes in. Those skip the model call entirely and trade correctness for speed. Prompt caching never reuses an answer; it only reuses the processed input, and the model still runs and generates a fresh response every time. This guide is strictly about prompt caching.

Caching lowers the cost and latency of input processing. It does nothing for output. That is a hard ceiling: if your workload’s cost is dominated by long generations rather than long prompts, caching the input is the wrong lever — a point the guide on why cutting generated code barely lowers your AI bill approaches from the opposite direction. Now that we know what gets cached, the question is how the money works.

The economics: write premium, read discount, and TTL

A cached request is priced differently from an ordinary one, and three mechanics account for all of it.

A cache write is the first request that stores a prefix. A cache read is a later request that reuses it. The read discount is the good news: a cache read is billed at a small fraction of the normal input rate. The write premium is the catch: on some providers, that first storing request costs more than an ordinary input token. And a cache entry does not live forever — it has a TTL (time to live), after which it is evicted and the next request has to write it again. Below a provider-specific minimum cacheable prefix, nothing is cached at all.

To make this concrete, here is the price ladder for one model, dated because these numbers drift and you should reverify them against the provider before you plan around them. As of July 2, 2026, Anthropic’s prompt caching documentation prices Claude Opus 4.8 like this:

Token treatment	Rate (per million tokens)	Relative to normal input
Normal input	$5.00	1×
Cache write, 5-minute TTL	$6.25	1.25×
Cache write, 1-hour TTL	$10.00	2×
Cache read	$0.50	0.1×
Output (unchanged by caching)	$25.00	—

Read that ladder top to bottom and the bet is visible in the numbers. A cache read is a tenth of normal input — that is where savings come from. But a cache write costs more than normal input, and the longer you want the entry to survive, the more the write costs. Output sits off to the side, untouched. You pay a premium up front to store the prefix, and you only come out ahead if enough cheap reads follow before the entry expires.

The write premium is not universal, and the difference between providers is the most useful thing to understand here, because it changes how the bet is scored:

• No write premium (automatic caching). OpenAI’s prompt caching is automatic — no code change, no breakpoint markers — and charges no extra fee to write a cache entry, only a discount to read one. When there is no write premium, the downside of a cache miss is simply that you did not get the discount; you did not lose money by caching.
• A write premium (explicit caching). Anthropic and Amazon Bedrock use explicit caching, where you mark where the cacheable prefix ends, and the write is billed above the normal input rate. Bedrock’s documentation describes the write as charged “at a rate higher than uncached input.” Here a miss has a real cost: you paid the premium and got no reads back.
• Storage rent (explicit caching, priced by time). Google Gemini’s context caching offers an explicit mode that, instead of a per-write multiplier, bills storage for as long as the cache lives — a rate per token per hour. The longer you hold the prefix, the more you pay to keep it, whether or not you read it.

Three ways to price the same idea: no premium, a write multiplier, or storage rent. In every case the structure is identical — you pay something to establish a reusable prefix and save on every reuse. With the mechanics in hand, the question becomes: when do these net out positive?

The mental model: caching is a workload-shape bet

Here is the one idea to carry out of this guide. Prompt caching pays off when the reads per write on your stable prefix are high enough to amortize the write premium before the entry’s TTL expires. That is it. Everything practical follows from three levers inside that sentence:

1. Prefix size — how much stable context sits in front of the volatile tail. A bigger stable prefix means each cache read saves more, and the write premium is spread over more tokens.
2. Reuse ratio (reads per write) — how many times you read a stored prefix before it changes or expires. One read per write rarely pays; many reads per write is where caching shines.
3. Reuse interval versus TTL — how the time between reuses compares to the cache lifetime. Reuse faster than the TTL and the entry stays warm; reuse slower and it expires between calls, so every request pays to write again and you never reach the discount.

The break-even is easy to reason about without heavy math. A cache write costs a little more than a normal input; a cache read costs a small fraction of it. So the first cached request is more expensive than not caching at all — you paid the premium and have not read anything back yet. Each subsequent read is far cheaper than reprocessing the prefix. Somewhere after the first read, the cumulative cost of the caching path drops below the cumulative cost of the no-caching path, and from there the gap widens with every reuse. The TTL sets a deadline on how many reads you are allowed to bank before the entry disappears.

With that model in hand, we can look at real workloads and say which side of the break-even they land on.

Where caching wins and where it loses

The model turns into a quick classification test. Look at a workload and ask: Is there a large stable prefix? Is it read many times per write? Does the reuse happen inside the TTL? The more “yes” answers, the more caching helps.

Workloads that win share a fat, stable prefix hit many times in quick succession:

• Coding agents carry a fixed system prompt, a large set of tool definitions, and repository context across dozens of steps in a single session. The prefix is enormous and reused on nearly every step, often several times a minute — a high reuse ratio well inside any TTL.
• Retrieval systems that answer many questions against the same retrieved document set reuse that document prefix across every question in the batch.
• Multi-turn assistants re-send the growing conversation as a prefix on each turn, and turns within an active session usually land inside the TTL.
• Batch classification with a shared instruction-and-examples preamble reads that preamble once per item across the whole run.

Workloads that lose or barely move are the mirror image:

• One-shot, varied prompts with no repeated prefix have nothing to reuse. On a write-premium provider, marking them cacheable only pays the premium for a read that never comes.
• Prefixes below the minimum cacheable size are silently not cached, so you get no discount no matter how often you call.
• Reuse slower than the TTL means the entry expires between calls; every request writes again and none of them read.
• Prefixes that change on every call — a timestamp near the front, a reshuffled system prompt, per-request personalization ahead of the stable content — break the prefix match and force a fresh write each time.

This is exactly the two-team split from the opening. The team whose cost dropped had a coding-agent-shaped workload: a big prefix, many reads per write, fast reuse. The team whose bill went up first and barely recovered had varied one-shot prompts on a write-premium provider — they paid to write entries that were rarely read.

You do not have to take the archetypes on faith. On July 2, 2026 we measured a deliberately simple version of the winning case: the same document held in the prefix, three different questions asked against it, run once with caching off and once with caching on, priced from the provider’s own reported usage on Claude Opus 4.8. The document was about 19,179 tokens — far above any provider’s minimum cacheable prefix — held in the cacheable position while only the question varied. You can read the exact script we ran in the cost-demo script.

On the provider we measured there is no separate “upload the document” step. In both runs the full document is sent inline at the front of every request — caching does not move it anywhere or store it ahead of time. On the caching run you only mark that prefix as cacheable; the provider then stores the processed prefix as a side effect of the first request that carries it. That is why the first row below is labelled “writes the prefix”: the write is billed as part of Question 1, not as a distinct call. Questions 2 and 3 send the same document again, but the provider matches the prefix it already holds and bills those tokens at the read rate instead of reprocessing them.

Not every provider works this inline way. Google Gemini’s explicit context caching does have a separate step: you create a cache with its own API call, get back a handle, and reference that handle on later requests — closer to the “upload it first” mental model. The break-even reasoning is the same either way; what differs is only how the cache is established and, as the pricing ladder above showed, how it is billed.

Run	Request	Cost
No caching	Question 1	$0.0989
	Question 2	$0.0996
	Question 3	$0.1003
	Total	$0.2988
Caching	Question 1 (writes the prefix)	$0.1236
	Question 2 (reads the prefix)	$0.0137
	Question 3 (reads the prefix)	$0.0129
	Total	$0.1502

Three things stand out in that table. First, the caching run’s first question cost more than the no-caching run’s first question — $0.1236 against $0.0989 — because it paid the write premium on the full document with no read yet to amortize it. A single question over this document would have lost money by caching. Second, the reads that followed cost roughly a tenth as much, because the ~19,000-token prefix was billed at the read rate instead of full input. Third, the break-even landed between the first and second question, and by the third the caching path was far ahead — $0.1502 against $0.2988, about half the cost over three questions, and the gap would keep widening with each further question against the same cached document.

Note that this write-premium-on-the-first-call effect is specific to explicit caching with a write premium. On an automatic, no-premium provider the first call would not cost extra — there is no premium to pay — so the downside of a bad bet is smaller there. The numbers above are dated and model-specific; treat them as a worked illustration of the shape, not a promise for your provider. Knowing which side of the line a workload sits on, the next question is how to move a losing workload toward winning.

Engineering the workload to cache well

Cache effectiveness is not something a provider hands you. It is a property of how you structure the request, and it is under your control. Every technique below is an application of the same model: grow the stable prefix, raise the reuse ratio, and keep reuse inside the TTL.

• Put stable content first. Caching works on a leading prefix, so anything that changes per request should live at the end. System prompt, tool definitions, and shared documents go up front; the new user turn goes last.
• Maximize the stable prefix. The more unchanging context you can gather ahead of the volatile tail, the larger the cached region and the bigger each read’s discount.
• Place the cache breakpoint at the widest stable boundary. Where a provider uses explicit cache breakpoints, set the marker at the last token that is identical across requests — not earlier, which wastes cacheable content, and not inside the volatile region, which breaks the match.
• Separate volatile content to the tail. A single per-request value near the front — a timestamp, a request id, a personalized greeting — changes the prefix and forces a fresh write. Move those to the tail so the prefix stays byte-stable.
• Cluster requests inside the TTL. Reuse has to beat the clock. Batching or grouping requests that share a prefix so they land within the TTL window keeps the entry warm and turns writes into reads.
• Watch the minimum-prefix floor. Below the provider’s minimum cacheable size nothing is stored, so very small prefixes get no benefit no matter how often they repeat.

The single most common source of “caching didn’t help” is an unstable prefix: a value that changes near the front of the request and quietly breaks the match on every call. The reorder — stable content ahead, volatile content behind — is usually the change that moves the bill most. Once a workload is engineered to cache well, the last question is how you confirm it is actually working and where it sits among your other cost levers.

Measuring it, and where it fits in your cost strategy

Caching is a bet, and you do not have to guess whether you won it — providers report the split. The usage object on a response tells you how many input tokens were served from cache versus processed fresh. Anthropic returns cache_read_input_tokens and cache_creation_input_tokens; OpenAI reports cached_tokens under its prompt-token details; Bedrock returns cacheReadInputTokens and cacheWriteInputTokens. In the measurement above, the read count on questions two and three came back at 19,176 tokens — direct confirmation that the document prefix was stored once and reused, not reprocessed.

The account-level version of the same bet. Claude’s product dashboard rolls those per-response counters into a single card — a hit rate and an estimated-saved figure over a trailing window (Figure 5: an 87% hit rate and about a dollar saved over the last 7 days). It is the win condition of this guide stated as a summary: reads landing against stored prefixes, priced against the counterfactual of paying full input on every call, with the sparkline tracing the same break-even climb Figure 2 predicts. Two caveats keep it honest. The saved figure is modeled — full-price cost minus what you actually paid, not a billed line item — so read it as direction, not accounting. And its hit rate is not the Console’s cache read ratio; reading Claude’s cache dashboard pins that denominator down, and the two screens count differently. The card tells you the bet is paying; the counters above tell you why.

Two signals tell you caching is working over time: the cached-token share of your input rising toward the stable-prefix fraction you expect, and the cost-per-request falling at steady usage. If you enabled caching and neither moved, the prefix is probably not matching — check for a volatile value near the front, a prefix below the minimum, or reuse slower than the TTL. Because caching changes what the bill and the token counts look like, this is where cost attribution matters; a workload can cut its real cost while its raw token count climbs, so you have to watch cost, not just tokens. The Hidden Economics of AI Coding guide makes that same point about reading a bill backward into the behavior that produced it, and the guide on tracking AI costs across providers covers pulling cached-versus-uncached usage together when it is scattered across several providers’ billing surfaces.

Caching also does not stand alone. It is one lever in a set — alongside routing requests to cheaper models by task and centralizing policy through a gateway, both covered in Hidden Economics — and the levers compose: a routed request can still cache its prefix, and a gateway can apply caching policy consistently across a fleet. Caching is usually the first one to reach for on a repeated-context workload, because it moves the bill most for the least added complexity, but it belongs inside a measured cost strategy rather than on its own. For the broader picture of controlling model spend, see LLM cost management.

Next steps

Prompt caching is a workload-shape bet: it saves money only when a large, stable prefix is reused enough times, within the cache’s TTL, to earn back its write premium. It caches input, not output; a cache write can cost more than an ordinary token; and on the wrong workload it can lose money outright. Once you can spot the stable prefix, estimate the reads per write against the TTL, and see which structural changes enlarge the cacheable region, you can predict the sign and rough size of the savings before writing a line of code.

From here:

• Enable it in your stack. How to enable prompt caching is the step-by-step companion to this guide: how to structure the prefix, turn caching on for each provider, confirm the cache-read counter, and measure the net saving. For the exact API and current pricing, each provider’s prompt-caching documentation — Anthropic, OpenAI, Google Gemini, and Amazon Bedrock — remains the primary source.
• Diagnose it when it isn’t working. If caching is on and the bill still hasn’t moved, Why your prompt cache isn’t working is the troubleshooting companion: it reads the provider’s counters as a decision tree — did the cache write, did it read, and did it actually save.
• Read it off the Claude Console. How to read Claude’s cache usage dashboard runs that same diagnosis without instrumenting your code — the read ratio, write amortization, and cache-diagnostics panels read straight off Anthropic’s screen.
• See it play out in the real world. What the OpenClaw ban taught everyone about the real cost of AI is the case study behind this economics: a whole tool got cut off because its cache misses broke the arithmetic a flat plan was priced on.
• See the wider lever set. The Hidden Economics of AI Coding places caching alongside routing, gateways, and observability.
• Measure the result. Tracking AI costs across providers and LLM cost management cover watching cached-token share and cost-per-request once caching is on.

The switch was never the decision. The workload shape is. Get that right and caching stops being a hopeful toggle and becomes a cost lever you can reason about.