OpenAI makes GPT-6 prompt caching visible and configurable

OpenAI has upgraded prompt caching for GPT-6 with higher default hit rates, explicit cache controls, a usage dashboard, and diagnostics for cache misses, according to OpenAI’s announcement. The company says eligible cached input tokens can receive discounts of up to 90%, reducing the cost of persistent agents that make dozens or hundreds of related API calls.

Prompt caching reuses work from the beginning of an earlier request when a new request starts with the same token sequence. Long-running agents repeatedly send system instructions, tool schemas, and conversation history, so a stable prefix can reduce both input processing and latency. Changes near the beginning of that prefix reduce the amount available for reuse.

For GPT-6, eligible shared prefixes reused within a 30-minute window now qualify for cache discounts. The higher default hit rates require no configuration, while the new controls give developers more influence over which content remains reusable.

Cache misses become traceable

OpenAI’s new caching dashboard shows the share of input served from cache, charts hit rates over time, and separates cached and uncached tokens for each model. That visibility can reveal regressions after changes to prompts, tools, or agent configuration.

When a hit rate drops, the diagnostics guide describes an API that compares a request with a recent similar request. The response provides a structured miss reason and counts the affected tokens:

{
  "prompt_cache_diagnostics": {
    "type": "cache_miss",
    "reason": "tools_changed",
    "comparison_reusable_tokens": 5629,
    "cache_missed_tokens": 5629
  }
}

A reason such as tools_changed points to a modified or reordered tool schema. Because caching depends on an identical token prefix, an early change can invalidate reuse for every token that follows it. The reusable and missed token counts show the size of that impact.

Controls keep prefixes stable

The new controls focus on preserving stable prompt sections as an agent accumulates messages, changes reasoning settings, and selects tools:

• Explicit cache breakpoints. Mark selected prompt prefixes as reusable, giving the application clearer control over cache boundaries.
• Reasoning changes within a conversation. GPT-6 models can change reasoning effort between responses by appending a configuration_update while retaining the original request-level reasoning-effort setting. This allows a difficult step to use more reasoning without invalidating the existing cached prefix.
• Append-only tool changes. Keep tool definitions in a stable order and use allowed_tools to expose the relevant subset for each call. Set tool_choice to none when the model should call no tools. Append revised instructions at the end of the context so earlier content remains stable.
• Prewarming. Send shared instructions, tool definitions, or reference material at startup so the first user request can reuse an existing prefix. Teams should include the prewarm request when calculating total cost and latency.

Early users report higher hit rates

OpenAI’s launch materials include production and evaluation results from several customers:

These customer-reported figures cover different production and evaluation workloads. Each team should measure hit rate, latency, cache writes, and total input cost against its own traffic patterns.

A five-step rollout

1. Establish a baseline. Use the dashboard to record hit rates, cached token share, latency, and input cost by model over a representative period.
2. Stabilize the prefix. Place durable system instructions and tool schemas first. Append user queries, retrieved passages, and other volatile content later.
3. Control prompt changes. Keep tool definitions in a fixed order, add explicit breakpoints where useful, and gate tool availability with allowed_tools or tool_choice.
4. Prewarm shared contexts. Test startup calls for high-reuse prompts, then verify that subsequent savings cover the additional request.
5. Diagnose regressions. Query the diagnostics endpoint after a hit-rate drop and correlate its reason code with recent prompt, schema, or configuration changes.

Anthropic and Google also offer prompt caching, and developers have long used stable system prompts and deterministic tool ordering to improve reuse. OpenAI’s GPT-6 update formalizes those practices through supported controls, while the dashboard and diagnostics connect each cache miss to a concrete prompt change and its token cost.