{"id":667944,"date":"2025-05-08T21:26:35","date_gmt":"2025-05-08T18:26:35","guid":{"rendered":"https:\/\/en.buradabiliyorum.com\/google-launches-implicit-caching-to-make-accessing-its-latest-ai-models-cheaper\/"},"modified":"2025-05-08T21:26:35","modified_gmt":"2025-05-08T18:26:35","slug":"google-launches-implicit-caching-to-make-accessing-its-latest-ai-models-cheaper","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/google-launches-implicit-caching-to-make-accessing-its-latest-ai-models-cheaper\/","title":{"rendered":"Google launches ‘implicit caching’ to make accessing its latest AI models cheaper"},"content":{"rendered":"
\n

Google is rolling out a feature in its Gemini API that the company claims will make its latest AI models cheaper for third-party developers.<\/p>\n

Google calls the feature \u201cimplicit caching\u201d and says it can deliver 75% savings on \u201crepetitive context\u201d passed to models via the Gemini API. It supports Google\u2019s Gemini 2.5 Pro and 2.5 Flash models.<\/p>\n

That\u2019s likely to be welcome news<\/a> to developers as the cost of using frontier models continues to grow.<\/p>\n

\n
\n
\n

We just shipped implicit caching in the Gemini API, automatically enabling a 75% cost savings with the Gemini 2.5 models when your request hits a cache \ud83d\udea2<\/p>\n

We also lowered the min token required to hit caches to 1K on 2.5 Flash and 2K on 2.5 Pro!<\/p>\n

\u2014 Logan Kilpatrick (@OfficialLoganK) May 8, 2025<\/a><\/p><\/blockquote>\n<\/div>\n<\/figure>\n

Caching, a widely adopted practice in the AI industry, reuses frequently accessed or pre-computed data from models to cut down on computing requirements and cost. For example, caches can store answers to questions users often ask of a model, eliminating the need for the model to recreate answers to the same request.<\/p>\n

Google previously offered model prompt caching, but only explicit<\/em> prompt caching, meaning devs had to define their highest-frequency prompts. While cost savings were supposed to be guaranteed, explicit prompt caching typically involved a lot of manual work.<\/p>\n

Some developers weren\u2019t pleased with how Google\u2019s explicit caching implementation worked for Gemini 2.5 Pro, which they said could cause surprisingly large API bills. Complaints reached a fever pitch in the past week, prompting the Gemini team to apologize<\/a> and pledge to make changes.<\/p>\n

In contrast to explicit caching, implicit caching is automatic. Enabled by default for Gemini 2.5 models, it passes on cost savings if a Gemini API request to a model hits a cache. <\/p>\n

\n
\n

Techcrunch event<\/p>\n

\n

\n\t\t\t\t\t\t\t\t\tBerkeley, CA<\/span>
\n\t\t\t\t\t\t\t\t\t\t\t\t\t|<\/span>
\n\t\t\t\t\t\t\t\t\t\t\t\t\tJune 5<\/span>\n\t\t\t\t\t\t\t<\/p>\n

\t\t\t\t\tBOOK NOW<\/span><\/p><\/div>\n<\/p><\/div>\n<\/div>\n

\u201c[W]hen you send a request to one of the Gemini 2.5 models, if the request shares a common prefix as one of previous requests, then it\u2019s eligible for a cache hit,\u201d explained Google in a blog post<\/a>. \u201cWe will dynamically pass cost savings back to you.\u201d<\/p>\n

The minimum prompt token count for implicit caching is 1,024 for 2.5 Flash and 2,048 for 2.5 Pro, according to Google\u2019s developer documentation<\/a>, which is not a terribly big amount, meaning it shouldn\u2019t take much to trigger these automatic savings. Tokens are the raw bits of data models work with, with a thousand tokens equivalent to about 750 words.<\/p>\n

Given that Google\u2019s last claims of cost savings from caching ran afoul, there are some buyer-beware areas in these new claims. For one, Google recommends that developers keep repetitive context at the beginning of requests to increase the chances of implicit cache hits. Context that might change from request to request should be app<\/a>ended at the end, the company says.<\/p>\n

For another, Google didn\u2019t offer any third-party verification that the new implicit caching system would deliver the promised automatic savings. So we\u2019ll have to see what early adopters say.<\/p>\n<\/div>\n