Skip to main content
The response cache answers an exact repeat of a request, streaming or not, with the response Aixy returned earlier, without calling the provider. It suits evaluation suites, retried batch jobs and deterministic agents that send identical requests.

Buy the add-on

The response cache is a paid add-on for Pro and Enterprise subscriptions billed directly by Aixy. Owners and organization admins buy it under LLMs gateway → Caching → Aixy response cache:
  1. Choose a capacity from 1 to 50 GB with the slider. The first GB is free; the dashboard shows the monthly price for the capacity you choose, excluding VAT.
  2. Accept the processing statement: Aixy stores encrypted copies of model responses in the EU for the lifetime you set, at most 24 hours, and erases them when the add-on ends.
  3. Activate it. The free gigabyte charges nothing. For more capacity, the prorated amount for the rest of the billing period is charged to your payment method on file; if the payment fails, nothing changes.
You can raise the capacity at any time; the difference is charged immediately. Lowering it or cancelling credits the unused time on your next invoice. Cancelling stops serving cached responses immediately and erases them. Your cache policies are kept.

Turn the cache on

With the add-on, set an organization policy, and optionally a project policy, to On with a lifetime from 1 minute to 24 hours. A project policy replaces the organization policy, so a project can opt out. Any policy change makes earlier entries unreachable.

What is reused

A request is a repeat only when the model reference, protocol, compatibility headers and the complete request body after guardrails are identical, within the same organization and project. JSON key order does not matter; every value does. A streaming request and the same request without streaming are different requests. Token counting, decision APIs, Responses requests with conversation or background, and requests handled by an inference hook are never cached. Only complete, successful responses up to 2 MiB are stored; a stream that is interrupted or fails is not. A repeat of a streaming request returns the stored event stream at once instead of token by token. A conversation that grows with every turn sends a different request each time, so only identical turns are reused. A repeat returns the earlier response, including its sampling at non-zero temperatures and any tool results. Responses carry x-aixy-response-cache: hit when served from the cache and x-aixy-response-cache: miss when eligible but not found. Activity shows the same.

Billing and limits

A cached response counts as a model request toward your monthly allowance. It has no provider cost and does not consume hard budgets, token or request rate limits. When the purchased capacity is full, new responses are not stored until older ones expire; nothing is evicted.

See what the cache saves

The Aixy response cache view shows, for the selected period and project, the requests served from the cache, the hit rate among eligible requests, and the provider spend avoided. The avoided spend is estimated from the provider cost of each original response; hits whose original response had no price are counted but not valued. The view also shows the capacity in use. Provider prompt caching, which providers bill at a lower rate for repeated prompt prefixes, has its own view: Provider prompt caching.