Prompt caching is a bet, not a switch

When the AI bill climbs, the usual advice is to turn on prompt caching. It is usually good advice, but “cache your prompts to save money” hides the one fact that decides whether it helps you at all.

Here is how it goes wrong. Two teams run the same kind of workload on the same model. Both enable prompt caching with the same one-line change. One team’s cost per request drops sharply. The other team’s bill barely moves, and for a short window, right after they turn it on, it goes up.

That happens because prompt caching is a bet. You are wagering that a large, unchanging piece of your context will be reused enough times, soon enough, to earn back the cost of storing it. For some workloads that bet is nearly free money. For others it never pays off. You can tell which one you have before you touch a dashboard.

The Hidden Economics of AI Coding guide makes the case that caching is the highest-leverage first move for repeated-context workloads, and notes that it “isn’t magic, just an engineering tool that pays off when the workload has reusable structure.” Here is what gets cached, why a cache write can cost more than an ordinary token, and how to predict the savings on a workload.

What prompt caching actually caches

Prompt caching caches your input, not the model’s output. Most of the confusion around caching traces back to missing that, so start there.

When you send a request to a language model, the provider has to process every token of the prompt before it can generate a single token of a reply. For a coding agent, a retrieval system, or a long chat, most of those input tokens are the same on every call: the system prompt, the tool definitions, the retrieved documents, the conversation so far. Only the tail end changes: the new user turn, the latest tool result. Prompt caching lets the provider store the processed form of that repeated leading section and charge a reduced price to reuse it, instead of reprocessing it from scratch each time.

“Prefix” here means the beginning of the request. The model reads your prompt in order, top to bottom, and the provider matches an incoming request against its cache the same way, starting at the first token and stopping at the first token that differs. The stable prefix is the stretch from token one up to that first point of change: everything at the front of the request that is identical on every call. It is the only part eligible to be cached, which is why the technique cares about where content sits. Repeated content buried behind something that changes earns nothing. The volatile tail is everything after that point, the part that changes from request to request. Caching operates on the prefix. The tail is always processed fresh, and the model’s generated output is never cached. It is produced new every time and billed at the normal output rate.

This is also why prompt caching is a different technique from semantic caching or response caching, which store a finished answer and return it when a similar question comes in. Those skip the model call entirely and trade correctness for speed. Prompt caching never reuses an answer; it only reuses the processed input, and the model still runs and generates a fresh response every time.

Caching lowers the cost and latency of input processing. It does nothing for output. That is a hard ceiling: if your workload’s cost is dominated by long generations rather than long prompts, caching the input is the wrong lever, a point the guide on why cutting generated code barely lowers your AI bill approaches from the opposite direction.

The economics: write premium, read discount, and TTL

A cached request is priced differently from an ordinary one, and three mechanics account for all of it.

A cache write is the first request that stores a prefix. A cache read is a later request that reuses it. The read discount is the good news: a cache read is billed at a small fraction of the normal input rate. The write premium is the catch: on some providers, that first storing request costs more than an ordinary input token. And a cache entry does not live forever. It has a TTL (time to live), after which it is evicted and the next request has to write it again. Below a provider-specific minimum cacheable prefix, nothing is cached at all.

To make this concrete, here is the price ladder for one model, dated because these numbers drift and you should reverify them against the provider before you plan around them. As of July 2, 2026, Anthropic’s prompt caching documentation prices Claude Opus 4.8 like this:

That Opus 4.8 ladder remains a useful dated example, but the read multiplier is no longer universal across Claude. On September 1, 2026, Claude Fable 5.1 and Mythos 5.1 launched at $10 per million normal input tokens and $0.25 per million cache-read tokens: 0.025× input, or one-fortieth rather than one-tenth. On September 22, 2026, Claude Opus 5.5 landed on a third rung, $4 per million normal input and $0.20 per million cache read, or 0.05×, while also cutting the base input and output rates its Opus 5 predecessor charged. Sonnet 5.5 joined that rung on September 28, 2026, reading cache at $0.10 per million against $2 input. Claude Haiku 5.5, released October 7, 2026, stayed on the standard one-tenth: $0.01 per million against $0.10 input for prompts up to 100,000 tokens. Across every rung the five-minute and one-hour write multipliers remain 1.25× and 2×. Cheaper reads move break-even earlier. The saving still depends on how much stable context is read again before the entry expires, so check the multiplier for the exact model you are calling.

You can see it in the numbers. A cache read is a fraction of normal input, and that is where savings come from. On models with a write premium, a cache write costs more than normal input. Output sits off to the side, untouched. You pay a premium up front to store the prefix, and you only come out ahead if enough cheap reads follow before the entry expires.

The write premium differs by provider:

• A write premium with implicit or explicit breakpoints. OpenAI’s GPT-5.6 and later models can place breakpoints automatically or use ones you choose. A write costs 1.25× ordinary input; a read costs 0.1× on most of them and 0.05× on GPT-6.1 Sol. An automatic breakpoint does not make the write free, so repeatedly caching changing content can cost more.
• No separate write premium on earlier models. Older OpenAI models support implicit caching but do not publish the GPT-5.6-and-later cache-write tier. Check the exact model’s price list before treating a cache miss as a premium charge.
• A write premium, placed automatically or by marker. Anthropic’s prompt caching can place the breakpoint for you from a single top-level cache_control field, or you can mark where the cacheable prefix ends yourself. Either way a write costs 1.25× or 2× the input rate, depending on the TTL. Amazon Bedrock caches Claude prompts implicitly, on a best-effort basis, and offers a marker (cachePoint in the Converse API) for explicit control. Bedrock’s documentation says that, depending on the model, cache writes “can be billed at a rate that is higher than the standard input token rate.” Where that premium applies, a miss has a real cost: you paid the premium and got no reads back.
• Storage rent (explicit caching, priced by time). Google Gemini’s context caching offers an explicit mode that, instead of a per-write multiplier, bills storage for as long as the cache lives, a rate per token per hour. The longer you hold the prefix, the more you pay to keep it, whether or not you read it.

Three ways to price the same idea: no separate write premium, a write multiplier, or storage rent. In each case, compare the cost of creating a reusable prefix with the savings from later reads.

The mental model: caching is a bet on your workload

Prompt caching pays off when the reads per write on your stable prefix are high enough to amortize the write premium before the entry’s TTL expires. Everything practical follows from three levers inside that sentence:

1. Prefix size: how much stable context sits in front of the volatile tail. A bigger stable prefix means each cache read saves more, and the write premium is spread over more tokens.
2. Reuse ratio (reads per write): how many times you read a stored prefix before it changes or expires. One read per write rarely pays; many reads per write is where caching shines.
3. Reuse interval versus TTL: how the time between reuses compares to the cache lifetime. Reuse faster than the TTL and the entry stays warm; reuse slower and it expires between calls, so every request pays to write again and you never reach the discount.

The break-even is easy to reason about without heavy math on a model that charges a write premium. A cache write costs a little more than normal input; a cache read costs a small fraction of it. So the first cached request is more expensive than not caching at all, because you paid the premium and have not read anything back yet. Each subsequent read is far cheaper than reprocessing the prefix. Somewhere after the first read, the cumulative cost of the caching path drops below the cumulative cost of the no-caching path, and from there the gap widens with every reuse. The TTL sets a deadline on how many reads you are allowed to bank before the entry disappears.

Where caching wins and where it loses

The model turns into a quick classification test. Look at a workload and ask: Is there a large stable prefix? Is it read many times per write? Does the reuse happen inside the TTL? The more “yes” answers, the more caching helps.

Workloads that win share a fat, stable prefix hit many times in quick succession:

• Coding agents carry a fixed system prompt, a large set of tool definitions, and repository context across dozens of steps in a single session. The prefix is enormous and reused on nearly every step, often several times a minute, a high reuse ratio well inside any TTL.
• Retrieval systems that answer many questions against the same retrieved document set reuse that document prefix across every question in the batch.
• Multi-turn assistants re-send the growing conversation as a prefix on each turn, and turns within an active session usually land inside the TTL.
• Batch classification with a shared instruction-and-examples preamble reads that preamble once per item across the whole run.

Workloads that lose or barely move are the mirror image:

• One-shot, varied prompts with no repeated prefix have nothing to reuse. On a write-premium provider, caching them, by marker or automatically, only pays the premium for a read that never comes.
• Prefixes below the minimum cacheable size are silently not cached, so you get no discount no matter how often you call.
• Reuse slower than the TTL means the entry expires between calls; every request writes again and none of them read.
• Prefixes that change on every call: a timestamp near the front, a reshuffled system prompt, per-request personalization ahead of the stable content, break the prefix match and force a fresh write each time.

This is the two-team split from the opening. The team whose cost dropped had a workload like a coding agent’s: a big prefix, many reads per write, fast reuse. The team whose bill went up first and barely recovered had varied one-shot prompts on a write-premium provider. They paid to write entries that were rarely read.

You do not have to take these examples on faith. On July 2, 2026 we measured a deliberately simple version of the winning case: the same document held in the prefix, three different questions asked against it, run once with caching off and once with caching on, priced from the provider’s own reported usage on Claude Opus 4.8. The document was about 19,179 tokens, far above any provider’s minimum cacheable prefix, held in the cacheable position while only the question varied. You can read the exact script we ran in the cost-demo script.

On the provider we measured there is no separate “upload the document” step. In both runs the full document is sent inline at the front of every request, and caching does not move it anywhere or store it ahead of time. On the caching run you only mark that prefix as cacheable; the provider then stores the processed prefix as a side effect of the first request that carries it. That is why the first row below is labelled “writes the prefix”: the write is billed as part of Question 1, not as a distinct call. Questions 2 and 3 send the same document again, but the provider matches the prefix it already holds and bills those tokens at the read rate instead of reprocessing them.

Not every provider works this inline way. Google Gemini’s explicit context caching does have a separate step: you create a cache with its own API call, get back a handle, and reference that handle on later requests, closer to the “upload it first” mental model. The break-even reasoning is the same either way; what differs is only how the cache is established and, as the pricing ladder above showed, how it is billed.

First, the caching run’s first question cost more than the no-caching run’s first question, $0.1236 against $0.0989, because it paid the write premium on the full document with no read yet to amortize it. A single question over this document would have lost money by caching. Second, the reads that followed cost roughly a tenth as much, because the ~19,000-token prefix was billed at the read rate instead of full input. Third, the break-even landed between the first and second question, and by the third the caching path was far ahead at $0.1502 against $0.2988, about half the cost over three questions, and the gap would keep widening with each further question against the same cached document.

The write-premium-on-the-first-call effect also applies to OpenAI’s GPT-5.6 and later models whenever they write a cache entry, in both implicit and explicit modes. The numbers above are dated and model-specific; treat them as a worked example, and expect your provider’s numbers to differ.

Engineering the workload to cache well

Whether caching pays depends on how you structure the request, which is under your control. Every technique below is an application of the same model: grow the stable prefix, raise the reuse ratio, and keep reuse inside the TTL.

• Put stable content first. Caching works on a leading prefix, so anything that changes per request should live at the end. System prompt, tool definitions, and shared documents go up front; the new user turn goes last.
• Maximize the stable prefix. The more unchanging context you can gather ahead of the volatile tail, the larger the cached region and the bigger each read’s discount.
• Place the cache breakpoint at the widest stable boundary. Where a provider uses explicit cache breakpoints, set the marker at the last token that is identical across requests. Not earlier, which wastes cacheable content, and not inside the volatile region, which breaks the match.
• Separate volatile content to the tail. A single per-request value near the front (a timestamp, a request id, a personalized greeting) changes the prefix and forces a fresh write. Move those to the tail so the prefix stays byte-stable.
• Cluster requests inside the TTL. Reuse has to beat the clock. Batching or grouping requests that share a prefix so they land within the TTL window keeps the entry warm and turns writes into reads.
• Watch the minimum-prefix floor. Below the provider’s minimum cacheable size nothing is stored, so very small prefixes get no benefit no matter how often they repeat.

The single most common source of “caching didn’t help” is an unstable prefix: a value that changes near the front of the request and quietly breaks the match on every call. The reorder, stable content ahead and volatile content behind, is usually the change that moves the bill most. Once a workload is engineered to cache well, the last question is how you confirm it is working and where it sits among your other cost levers.

Measuring it, and where it fits in your cost strategy

You do not have to guess whether caching is working. Providers report the split. The usage object on a response tells you how many input tokens were served from cache versus processed fresh. Anthropic returns cache_read_input_tokens and cache_creation_input_tokens; OpenAI reports cached_tokens under its prompt-token details; Bedrock returns cacheReadInputTokens and cacheWriteInputTokens. In the measurement above, the read count on questions two and three came back at 19,176 tokens. That is direct confirmation that the document prefix was stored once and reused, not reprocessed.

The account-level view. Claude’s product dashboard rolls those per-response counters into a single card: a hit rate and an estimated-saved figure over a trailing window (Figure 5: an 87% hit rate and a small estimated saving over the last 7 days). It is the win condition of this guide stated as a summary: reads landing against stored prefixes, priced against the counterfactual of paying full input on every call, with the sparkline tracing the same break-even climb Figure 2 predicts. The saved figure is modeled, full-price cost minus what you paid, not a billed line item, so read it as a trend. And its hit rate is not the Console’s cache read ratio; reading Claude’s cache dashboard pins that denominator down, and the two screens count differently.

Two signals tell you caching is working over time: the cached-token share of your input rising toward the stable-prefix fraction you expect, and the cost-per-request falling at steady usage. If you enabled caching and neither moved, the prefix is probably not matching. Check for a volatile value near the front, a prefix below the minimum, or reuse slower than the TTL. Because caching changes what the bill and the token counts look like, this is where cost attribution matters; a workload can cut its real cost while its raw token count climbs, so watch cost. The Hidden Economics of AI Coding guide makes that same point about reading a bill backward into the behavior that produced it, and the guide on tracking AI costs across providers covers keeping one running total of billed spend when it is scattered across several providers’ billing pages.

Caching also does not stand alone. It is one lever in a set, alongside routing requests to cheaper models by task and centralizing policy through a gateway, both covered in Hidden Economics, and the levers compose: a routed request can still cache its prefix, and a gateway can apply caching policy consistently across a fleet. Caching is usually the first one to reach for on a repeated-context workload, because it moves the bill most for the least added complexity, but it belongs inside a measured cost strategy rather than on its own. For the broader picture of controlling model spend, see LLM cost management.

Next steps

Prompt caching saves money only when a large, stable prefix is reused enough times, within the cache’s TTL, to earn back its write premium. It caches input, not output; a cache write can cost more than an ordinary token; and on the wrong workload it can lose money outright. Once you can spot the stable prefix, estimate the reads per write against the TTL, and see which structural changes enlarge the cacheable region, you can predict the sign and rough size of the savings before writing a line of code.

From here:

• Enable it in your stack. How to enable prompt caching is the step-by-step companion to this guide: how to structure the prefix, turn caching on for each provider, confirm the cache-read counter, and measure the net saving. For the exact API and current pricing, each provider’s prompt-caching documentation, Anthropic, OpenAI, Google Gemini, and Amazon Bedrock remains the primary source.
• Diagnose it when it isn’t working. If caching is on and the bill still hasn’t moved, Why your prompt cache isn’t working is the troubleshooting companion: it reads the provider’s counters as a decision tree: did the cache write, did it read, and did it save.
• Read it off the Claude Console. How to read Claude’s cache usage dashboard runs that same diagnosis without instrumenting your code: the read ratio, write amortization, and cache-diagnostics panels read straight off Anthropic’s screen.
• See it play out in the real world. What the OpenClaw ban taught everyone about the real cost of AI is the case study behind this economics: a whole tool got cut off because its cache misses broke the arithmetic a flat plan was priced on.
• See the wider lever set. The Hidden Economics of AI Coding places caching alongside routing, gateways, and observability.
• Measure the result. Tracking AI costs across providers covers one running total of billed spend across providers, and LLM cost management covers reading spend by model, when you look, to check whether a change moved the bill.

The switch was never the decision. The workload is.