Prompt Caching with the Claude API: Cut Token Costs by 90% in TypeScript

The invoice nobody budgeted for

Your RAG assistant works. It retrieves the right documents, the answers are good, users are happy. Then the first full month of traffic lands and the API bill is four times what the prototype suggested.

Nothing broke. You are simply paying full price, on every single request, to re-send the same 40,000-token system prompt, the same tool definitions, and the same conversation history that the model already processed thirty seconds ago.

Prompt caching fixes exactly this. It is not a rewrite, not a new SDK, and not a different model — it is a field you add to a content block. But it is also easy to add in a way that does nothing at all, which is why most teams who "turned on caching" are still paying full price and do not know it.

This tutorial covers the mechanism honestly: how the prefix match works, why a single timestamp can silently disable everything, where to actually put the breakpoints, and how to prove the savings with real numbers rather than hope.

By the end you will have:

• A working Claude client in TypeScript with caching applied correctly
• A cached Next.js API route serving a large document context
• A cache-hit instrumentation helper that reports real token savings per request
• An audit checklist for the invalidators that silently break caching
• Multi-turn conversation caching that keeps paying off as the chat grows

Prerequisites

Before starting, make sure you have:

• Node.js 20+ installed
• TypeScript basics — interfaces, async/await, discriminated unions
• An Anthropic API key (set as ANTHROPIC_API_KEY) or an active profile from ant auth login
• Familiarity with calling an LLM API at least once — this is not an "intro to LLMs" guide
• Optional: a Next.js 15+ app if you want to follow the API-route section

Everything here uses the official @anthropic-ai/sdk package. No wrappers, no LangChain, no abstractions in between.

What you'll build

A document-QA endpoint that loads a large reference corpus into the system prompt once, then answers an unlimited number of user questions against it while paying roughly one tenth the input price for the shared portion.

Along the way we will build a small logCacheUsage helper that prints, per request, exactly how many tokens were written to cache, read from cache, and processed at full price — because caching you cannot measure is caching you cannot trust.

Step 1: The one rule everything else follows from

Read this section twice. Every strange caching bug traces back to it.

Prompt caching is a prefix match. Any change anywhere in the prefix invalidates everything after it.

The cache key is derived from the exact bytes of the rendered prompt up to each breakpoint. A single differing byte at position N invalidates the cache for every breakpoint at position N or later.

The API renders your request in a fixed order:

tools  →  system  →  messages

That ordering is the whole game. Tools render first, so changing a tool definition invalidates everything. The system prompt renders second, so a timestamp interpolated into the system prompt invalidates the entire conversation history that follows it.

This gives you a single design principle to follow for the rest of the tutorial:

Stable content first. Volatile content last.

If a piece of your prompt changes on every request, it must sit after your final cache breakpoint. If it sits before, nothing after it will ever cache, no matter how many cache_control markers you sprinkle around.

Step 2: Project setup

Create the project and install the SDK.

mkdir claude-caching-demo && cd claude-caching-demo
npm init -y
npm install @anthropic-ai/sdk
npm install -D typescript tsx @types/node
npx tsc --init

Set your credentials. Never hardcode the key.

export ANTHROPIC_API_KEY="sk-ant-..."

Create src/client.ts:

import Anthropic from "@anthropic-ai/sdk";
 
// The zero-arg constructor resolves credentials from the environment:
// ANTHROPIC_API_KEY, then ANTHROPIC_AUTH_TOKEN, then an `ant auth login` profile.
export const client = new Anthropic();
 
export const MODEL = "claude-opus-5";

A note on model choice that matters more than it looks: the minimum cacheable prefix differs by model, and it is not monotonic across generations.

A 3,000-token prompt caches on Claude Opus 5 and silently does not on Haiku 4.5. There is no error and no warning — you just get cache_creation_input_tokens: 0 and a normal-looking bill. If you migrate models and your hit rate drops to zero, check this table before debugging anything else.

Step 3: A baseline request with no caching

Start with the naive version so we have something to compare against. Create src/baseline.ts:

import { client, MODEL } from "./client";
import { readFileSync } from "fs";
 
// Imagine this is your product documentation, legal corpus,
// codebase summary — anything large and stable.
const REFERENCE_DOC = readFileSync("./data/handbook.md", "utf-8");
 
const SYSTEM_PROMPT = `You are a support assistant for Noqta.
Answer strictly from the reference document below.
If the answer is not in the document, say so plainly.
 
<reference_document>
${REFERENCE_DOC}
</reference_document>`;
 
async function ask(question: string) {
  const response = await client.messages.create({
    model: MODEL,
    max_tokens: 16000,
    system: SYSTEM_PROMPT,
    messages: [{ role: "user", content: question }],
  });
 
  for (const block of response.content) {
    if (block.type === "text") {
      console.log(block.text);
    }
  }
 
  console.log("--- usage ---", response.usage);
}
 
await ask("What is the refund window for annual plans?");
await ask("Do you support SSO on the team tier?");

Run it:

npx tsx src/baseline.ts

Both requests report a large input_tokens value and zero cache activity. If the handbook is 40,000 tokens, you paid for 40,000 input tokens twice to answer two unrelated one-line questions. Ten thousand questions per month means 400 million input tokens of pure repetition.

Note the discriminated-union narrowing in that loop. response.content is a ContentBlock[], and TypeScript will reject content[0].text without a block.type === "text" check. This is not caching-specific, but it trips up everyone on their first Claude request in TypeScript.

Step 4: Add caching — the simple way

The fastest correct fix is top-level automatic caching. Add one field.

Create src/cached-auto.ts:

import { client, MODEL } from "./client";
import { readFileSync } from "fs";
 
const REFERENCE_DOC = readFileSync("./data/handbook.md", "utf-8");
 
const SYSTEM_PROMPT = `You are a support assistant for Noqta.
Answer strictly from the reference document below.
If the answer is not in the document, say so plainly.
 
<reference_document>
${REFERENCE_DOC}
</reference_document>`;
 
async function ask(question: string) {
  const response = await client.messages.create({
    model: MODEL,
    max_tokens: 16000,
    // Auto-places the breakpoint on the last cacheable block.
    cache_control: { type: "ephemeral" },
    system: SYSTEM_PROMPT,
    messages: [{ role: "user", content: question }],
  });
 
  console.log("--- usage ---", response.usage);
  return response;
}
 
await ask("What is the refund window for annual plans?");
await ask("Do you support SSO on the team tier?");

Run it and look at the two usage objects:

--- usage --- { input_tokens: 14, cache_creation_input_tokens: 41203, cache_read_input_tokens: 0, output_tokens: 87 }
--- usage --- { input_tokens: 15, cache_creation_input_tokens: 0, cache_read_input_tokens: 41203, output_tokens: 64 }

That second line is the whole point. The first request wrote 41,203 tokens into the cache at roughly 1.25× the normal input rate. The second read them back at roughly 0.1×.

One field on the usage object is worth internalizing: input_tokens is the uncached remainder only, not the total prompt size. The real total is:

input_tokens + cache_creation_input_tokens + cache_read_input_tokens

If you have an agent that ran for two hours and input_tokens reports 14, that is not a bug and not a miracle — the rest was served from cache. Cost dashboards that chart input_tokens alone will quietly under-report your real usage.

Step 5: Manual breakpoints, and when you need them

Automatic caching places one breakpoint on the last cacheable block. That is right for the single-shared-context case above, and wrong for a very common one: a shared prefix with a varying suffix in the same message.

Consider a few-shot classifier where every request sends the same large example set plus a different input:

// WRONG — the breakpoint lands after the varying input,
// so every request writes a brand-new cache entry and never reads one.
const response = await client.messages.create({
  model: MODEL,
  max_tokens: 1024,
  cache_control: { type: "ephemeral" },
  messages: [
    {
      role: "user",
      content: [
        { type: "text", text: FEW_SHOT_EXAMPLES },
        { type: "text", text: `Classify this ticket: ${ticketBody}` },
      ],
    },
  ],
});

Every request produces a different final block, so every request keys a different cache entry. You pay the 1.25× write premium forever and read nothing back. This is the single most common way to make caching more expensive than not caching.

The fix is to mark the end of the shared portion explicitly:

// RIGHT — breakpoint on the shared examples, varying input after it.
const response = await client.messages.create({
  model: MODEL,
  max_tokens: 1024,
  messages: [
    {
      role: "user",
      content: [
        {
          type: "text",
          text: FEW_SHOT_EXAMPLES,
          cache_control: { type: "ephemeral" },
        },
        // No marker — this differs every time and must come last.
        { type: "text", text: `Classify this ticket: ${ticketBody}` },
      ],
    },
  ],
});

The rules for manual placement:

• Maximum 4 breakpoints per request
• Valid on system text blocks, tool definitions, and message content blocks (text, image, tool_use, tool_result, document)
• A breakpoint on the last system block caches tools and system together, because tools render first

Step 6: Build a cache instrumentation helper

Guessing at cache behaviour is how teams end up with a caching implementation that has never once produced a hit. Build the measurement first.

Create src/cache-usage.ts:

import type Anthropic from "@anthropic-ai/sdk";
 
// Approximate Claude Opus 5 rates, USD per million tokens.
const INPUT_RATE = 5.0;
const CACHE_WRITE_MULTIPLIER = 1.25; // 2.0 for the 1-hour TTL
const CACHE_READ_MULTIPLIER = 0.1;
 
export function logCacheUsage(label: string, usage: Anthropic.Usage) {
  const fresh = usage.input_tokens;
  const written = usage.cache_creation_input_tokens ?? 0;
  const read = usage.cache_read_input_tokens ?? 0;
  const total = fresh + written + read;
 
  const actual =
    (fresh +
      written * CACHE_WRITE_MULTIPLIER +
      read * CACHE_READ_MULTIPLIER) *
    (INPUT_RATE / 1_000_000);
 
  const uncached = total * (INPUT_RATE / 1_000_000);
  const saved = uncached - actual;
  const hitRate = total > 0 ? (read / total) * 100 : 0;
 
  console.log(
    [
      `[${label}]`,
      `total=${total}`,
      `fresh=${fresh}`,
      `written=${written}`,
      `read=${read}`,
      `hit=${hitRate.toFixed(1)}%`,
      `cost=$${actual.toFixed(5)}`,
      `saved=$${saved.toFixed(5)}`,
    ].join(" "),
  );
 
  return { total, fresh, written, read, hitRate, actual, saved };
}

Wire it into any call:

import { logCacheUsage } from "./cache-usage";
 
const response = await client.messages.create({ /* ... */ });
logCacheUsage("doc-qa", response.usage);

Now the diagnostic rule is concrete: if read stays at 0 across repeated requests with what should be an identical prefix, you have a silent invalidator. That is not a caching limitation — it is a bug in your prompt assembly, and the next step is how to find it.

Step 7: Audit for silent invalidators

These are the patterns that quietly disable caching. Grep your prompt-assembly code for every one of them.

The most common offender by far:

// This single line makes the entire conversation uncacheable.
const system = `You are a helpful assistant.
Current date: ${new Date().toISOString()}
 
${LARGE_STABLE_INSTRUCTIONS}`;

The date sits at the front of the prefix, so the 30,000 tokens of stable instructions after it are re-processed at full price every single time.

The fix is to move volatile context out of the system prompt entirely:

const system = [
  {
    type: "text" as const,
    text: LARGE_STABLE_INSTRUCTIONS,
    cache_control: { type: "ephemeral" as const },
  },
];
 
const messages = [
  ...history,
  {
    role: "user" as const,
    content: `Current date: ${new Date().toISOString()}\n\n${userQuestion}`,
  },
];

Same information, delivered to the model, with the cache intact.

Serialize tools deterministically

Tools render at position 0, which makes them the highest-leverage thing to keep stable:

// Sort by name so ordering never depends on object-key iteration or a Set.
const tools = Object.values(toolRegistry).sort((a, b) =>
  a.name.localeCompare(b.name),
);

Do not swap the tool set to implement "modes". If you need a mode switch, pass it as message content, or give the model a tool that records the transition — anything except mutating the tool array mid-conversation.

Step 8: Not everything invalidates everything

This is where a lot of over-caution comes from. Changing a request parameter does not automatically nuke the whole cache. There are three tiers, and a change only invalidates its own tier and the tiers below it.

The practical takeaway: you can flip tool_choice per request, or toggle thinking on and off, without losing the tools-plus-system cache. Only tool definition changes and model switches force a full rebuild.

Two of these rows have an escape hatch worth knowing:

• System prompt edits. On Claude Opus 5, Claude Opus 4.8, Claude Fable 5, and Claude Mythos 5, you can append a { role: "system", content: "..." } message to the messages array instead of editing the top-level system field. It sits after the cached history, so the prefix survives. No beta header required. It is also the injection-safe way to deliver operator instructions, since text inside a user turn can be forged by anything that writes to user input. Note this is not supported on Claude Sonnet 5 — that returns a 400.
• Tool set changes. From Claude Opus 5 onward, behind the mid-conversation-tool-changes-2026-07-01 beta header, you can add and remove tools between turns using tool_addition and tool_removal blocks without invalidating the cache.

Model switching has no escape hatch — caches are model-scoped. If you want a cheaper model for a sub-task, spawn a separate call for it and keep your main loop on one model.

Step 9: Caching multi-turn conversations

For a chat interface, the win compounds. Place the breakpoint on the last content block of the most recently added turn, and each request reuses the entire prior conversation.

Create src/conversation.ts:

import Anthropic from "@anthropic-ai/sdk";
import { client, MODEL } from "./client";
import { logCacheUsage } from "./cache-usage";
 
const SYSTEM_PROMPT = "You are a concise technical assistant.";
 
export class CachedConversation {
  private messages: Anthropic.MessageParam[] = [];
 
  async send(userText: string): Promise<string> {
    this.messages.push({
      role: "user",
      content: [
        {
          type: "text",
          text: userText,
          // Breakpoint on the newest turn: the whole prior
          // conversation becomes the reusable cached prefix.
          cache_control: { type: "ephemeral" },
        },
      ],
    });
 
    const response = await client.messages.create({
      model: MODEL,
      max_tokens: 16000,
      system: SYSTEM_PROMPT,
      messages: this.messages,
    });
 
    // Append the full content array, not just the text —
    // dropping blocks breaks tool use and thinking continuity.
    this.messages.push({ role: "assistant", content: response.content });
 
    logCacheUsage(`turn-${this.messages.length / 2}`, response.usage);
 
    return response.content
      .filter((b): b is Anthropic.TextBlock => b.type === "text")
      .map((b) => b.text)
      .join("");
  }
}

Run a few turns and watch read climb while fresh stays tiny. Earlier breakpoints remain valid read points, so hits accrue incrementally as the conversation grows rather than resetting each turn.

The 20-block lookback trap

Each breakpoint walks backward at most 20 content blocks looking for a prior cache entry. In an agentic loop, a single turn can easily add more than 20 blocks — every tool_use and tool_result pair counts.

When that happens, the next request's breakpoint cannot see the previous cache and silently misses. There is no error; your hit rate just collapses on long tool-heavy turns.

The fix is to place an intermediate breakpoint roughly every 15 blocks in long turns, keeping each marker within 20 blocks of the previous cached one. This is exactly the kind of failure that is invisible without the logCacheUsage helper from Step 6.

Step 10: A cached Next.js API route

Putting it together in a real endpoint. Create app/api/ask/route.ts:

import Anthropic from "@anthropic-ai/sdk";
import { NextResponse } from "next/server";
import { getHandbook } from "@/lib/handbook";
 
const client = new Anthropic();
 
export async function POST(request: Request) {
  const { question } = await request.json();
 
  if (typeof question !== "string" || question.trim().length === 0) {
    return NextResponse.json(
      { error: "A non-empty question is required." },
      { status: 400 },
    );
  }
 
  // Loaded once at module scope in getHandbook — byte-identical
  // across requests, which is what makes the cache hit.
  const handbook = await getHandbook();
 
  try {
    const response = await client.messages.create({
      model: "claude-opus-5",
      max_tokens: 4096,
      system: [
        {
          type: "text",
          text: `You are a support assistant. Answer only from the handbook below.\n\n<handbook>\n${handbook}\n</handbook>`,
          cache_control: { type: "ephemeral", ttl: "1h" },
        },
      ],
      messages: [{ role: "user", content: question }],
    });
 
    const answer = response.content
      .filter((b): b is Anthropic.TextBlock => b.type === "text")
      .map((b) => b.text)
      .join("");
 
    return NextResponse.json({
      answer,
      usage: {
        cached: response.usage.cache_read_input_tokens ?? 0,
        fresh: response.usage.input_tokens,
      },
    });
  } catch (error) {
    if (error instanceof Anthropic.RateLimitError) {
      return NextResponse.json(
        { error: "Rate limited. Retry shortly." },
        { status: 429 },
      );
    }
    if (error instanceof Anthropic.APIError) {
      return NextResponse.json(
        { error: `Upstream error: ${error.message}` },
        { status: 502 },
      );
    }
    throw error;
  }
}

Two details carry real weight here.

Typed error handling. Anthropic.RateLimitError and friends are exported classes with a typed status field. Check most-specific first, and never match on error message strings — those change without notice.

The 1-hour TTL. Notice ttl: "1h" on the cache_control. That choice deserves its own section.

Step 11: Choosing a TTL, honestly

The economics are simple enough to reason about directly:

Break-even follows from those numbers:

• 5-minute TTL: two requests break even. One write plus one read is 1.35× versus 2× uncached.
• 1-hour TTL: you need at least three requests. Two write premium plus two reads is 2.2× versus 3× uncached.

So the 1-hour TTL is not strictly better. It keeps entries alive across traffic gaps, which matters for bursty workloads, but the doubled write cost means it needs more reads to pay for itself. For a steadily-trafficked endpoint where requests arrive more often than every five minutes, the default 5-minute TTL is usually the cheaper choice — real traffic keeps the cache warm on its own.

Step 12: Two timing behaviours that surprise people

Concurrent requests all miss

A cache entry becomes readable only after the first response begins streaming. Fire ten parallel requests with an identical prefix and all ten pay full price — none can read what the others are still writing.

For fan-out patterns, sequence the first one:

// Send one request, await the first streamed token, then fan out.
const first = client.messages.stream({ /* ...shared prefix... */ });
for await (const _event of first) break; // wait for the stream to open
 
const rest = await Promise.all(
  remainingInputs.map((input) => client.messages.create({ /* ... */ })),
);

You wait for the first token, not the full response. The cache is live from that moment.

Pre-warming with max_tokens: 0

To remove the cold-start latency on the first real request, send a zero-output request at startup. The API runs prefill, writes the cache, and returns immediately with empty content:

await client.messages.create({
  model: MODEL,
  max_tokens: 0,
  system: [
    {
      type: "text",
      text: SYSTEM_PROMPT,
      cache_control: { type: "ephemeral" },
    },
  ],
  messages: [{ role: "user", content: "warmup" }],
});

You are billed the normal cache-write charge and zero output tokens. The response comes back with content: [] and stop_reason: "max_tokens".

Be deliberate about when this is worth it. Pre-warming trades a write charge now for lower time-to-first-token later. It pays off when all three hold: first-request latency is user-visible, the shared prefix is large, and there is a quiet moment before traffic — app startup, worker boot, post-deploy.

Skip it when traffic is continuous (real requests keep the cache warm for free), when the prefix varies per user (nothing shared to warm), or when you would be speculatively warming many distinct prefixes (each is a write you may never read).

Place the breakpoint on the last block shared with the real request — the system prompt or tool definitions — not on the placeholder user message, and do not use top-level automatic caching here, since that would key the cache to the throwaway placeholder.

max_tokens: 0 is rejected in combination with stream: true, enabled thinking, output_config.format, a forced tool_choice, or the Batches API.

Testing your implementation

Verification is straightforward because the API tells you the truth:

1. Run the same request twice. The second should show a non-zero cache_read_input_tokens. If it does not, you have an invalidator.
2. Check the sum. input_tokens + cache_creation_input_tokens + cache_read_input_tokens should approximately equal your full prompt size. If the sum is far below what you expect, you are not sending what you think you are sending.
3. Diff the rendered prompt. When a hit refuses to appear, serialize the full request body on two consecutive calls and diff the bytes. The offending timestamp or reordered key will be obvious.
4. Watch the hit rate over a real session. A chat that starts at 0% and climbs past 90% by turn five is behaving correctly. One that oscillates is hitting the 20-block lookback limit.

Troubleshooting

cache_read_input_tokens is always 0. The prefix differs between requests. Work through the Step 7 table. In practice it is a timestamp, a UUID, or a per-user ID inside the system prompt about 80% of the time.

cache_creation_input_tokens is 0 too, and no error appears. Your prefix is below the model's minimum cacheable length. Check the Step 2 table — the minimum ranges from 512 tokens on Claude Opus 5 up to 4096 on Opus 4.6 and Haiku 4.5.

Caching made my bill go up. You are writing without reading. This is the shared-prefix-with-varying-suffix mistake from Step 5 — the breakpoint is after the volatile content, so every request writes a fresh entry. Move the marker to the end of the shared portion.

Hit rate collapses on long agentic turns. The 20-block lookback window. Add intermediate breakpoints every ~15 blocks.

Hits work locally, fail in production. Usually concurrency. Parallel requests cannot read a cache that is still being written. Sequence the first request, or accept the miss on cold start.

Hits stopped after a deploy. Check whether the model string, the tool set, or the system prompt changed. All three invalidate. Caches are also model-scoped, so a model bump always starts cold.

Next steps

• Add token counting before requests. client.messages.countTokens() gives you exact, model-specific counts. Do not use tiktoken — that is OpenAI's tokenizer and undercounts Claude tokens by 15–20% on ordinary text, and far more on code.
• Combine with effort tuning. output_config: { effort: "low" | "medium" | "high" | "xhigh" | "max" } controls thinking depth and total token spend. Caching cuts input cost; effort cuts output cost. They are independent levers, and most teams only pull one.
• Instrument in production. Ship the Step 6 helper's output to your observability stack. A cache hit rate chart catches a prompt-assembly regression the same day it lands, instead of on the invoice.
• Explore related tutorials on noqta.tn: the Claude Agent SDK TypeScript guide for agent loops that benefit heavily from caching, LLM observability with Langfuse for tracing token spend end to end, and Vercel AI Gateway routing for multi-provider setups.

Conclusion

Prompt caching is one of the rare optimizations that is both large and cheap to adopt. A correctly placed breakpoint on a stable prefix takes about ten minutes to implement and reduces the input cost of the shared portion by roughly 90%.

The catch is that "correctly placed" carries all the weight. The mechanism is a byte-exact prefix match, so a single interpolated timestamp, an unsorted JSON serialization, or a breakpoint placed one block too late turns a 90% saving into a 25% surcharge — silently, with no error and no warning.

Which is why the instrumentation in Step 6 is not an optional extra. Log cache_read_input_tokens on every request, alert when it drops, and treat a falling hit rate as the regression it is. The teams that get real value from caching are not the ones who enabled it — they are the ones who can prove, today, that it is still working.