Caching

Opt-in, always

Nothing is cached unless the request asks for it. A gateway that cached by default would silently make every application non-deterministic: the same call returning a frozen answer an hour later, with no way to tell from the outside that it had.

Ask for it with a header carrying the number of seconds you want the answer kept:

curl https://api.multigrid.ai/v1/chat/completions \
  -H "Authorization: Bearer $MULTIGRID_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-Multigrid-Cache-Ttl: 3600" \
  -d '{
    "model": "openai/gpt-5-nano",
    "messages": [{"role": "system", "content": "Classify as billing, bug or feature."},
                 {"role": "user", "content": "My card was charged twice."}]
  }'

# A hit answers with:
#   X-Multigrid-Cache: hit
#   X-Multigrid-Cost-Usd: 0.000000
#   "multigrid": { "cache": "hit", "cost_micros": 0, "saved_usd": 0.000021, … }

A hit costs nothing, reaches no provider, and reports what it saved. A streaming caller gets a hit replayed as SSEStreamingThe answer arrives word by word as it is written, instead of all at once at the end. It costs the same; it just feels far faster., because their SDK is already in streaming mode and would choke on anything else, in one chunk, since pretending to type it out slowly would be theatre.

What makes two requests the same

The cache key is a SHA-256 over your account id and everything in the request body that could change the answer. It is a deny-list: every field is part of the key unless it is named below, so a sampling parameter we have never heard of still makes a different question.

Left out, and only these:

• stream and stream_options — the same question streamed and unstreamed has the same answer, and leaving them out lets a streaming caller benefit from a non-streaming caller’s entry.
• user and metadata — who asked and how you label it, not what comes back.
• cache and retry — the directives themselves. A 60-second TTL and a 600-second one are one question.
• max_tokens and max_completion_tokens — folded into one ceiling under one name rather than dropped, so moving from the deprecated spelling to the current one does not lose every entry you had.
• The ordering half of provider (order, sort, weights): every route they choose between can serve the request, so which one won changes latency and price, not the answer. That is also what lets a :cheap request and a :fast one share one entry. The filtering half — ignore and allow_fallbacks — is in the key, because a provider you forbade is not one a cached answer may quietly come from.

stream is deliberately not part of the key: the same question streamed and unstreamed has the same answer, so a streaming caller can be served from a non-streaming caller’s entry. Your user field and routing hints are excluded too. They change who asked and where it ran, not what comes back.

Matching is exact. Two prompts differing by one character are two different requests.

Rules and limits

Hit counts and the last hit time are kept per entry, and the Cache page shows what is live and what it has saved you.

Provider-side prompt caching

This is a different thing that you also benefit from, and it is not something you switch on here. Several providers cache the leading portion of a prompt on their own side and charge a fraction of the normal rate for it. When they report that, the count arrives as usage.prompt_tokens_details.cached_tokens and those tokens are billed at the route’s cached rate rather than the full prompt rate.

So: the cache above skips the provider entirely and costs nothing; prompt caching still calls the provider and costs less. Both show up on the same response, and only the first is yours to control.