Cloudflare has added native web search to AI Gateway, its proxy for AI model traffic. The announcement on October 2, 2026 lets developers inject real-time web results into model inference calls without building their own search layer.
What it does
AI models answer from what they learned in training unless something gives them fresher context. The Web Search API supplies that context. Developers can call it through AI Gateway, through REST APIs, or through Workers bindings, and the search results flow into the model request.
The launch partners are three search providers built for AI use: Ceramic.ai, Exa, and Linkup. Cloudflare says the integration does not retain the data that passes through it.
The crawler rules
The more interesting part for site owners is the policy attached to it. Cloudflare says the search providers it works with must follow specific rules that respect site owners' choices about crawling. The company has spent the last two years building tools that let sites block, allow, or charge AI crawlers, and it is now positioning its own search integration as an example of what it considers responsible bot behavior.
Why it matters for AEO
Every new search layer for AI is another route by which your pages can end up in an answer. A few practical points:
- Search APIs for AI are multiplying. Perplexity launched a cheaper Fast Search tier the week before. Developers now have several ways to give models live web results, and each provider runs its own index and crawler.
- Your crawler policy decides your eligibility. If your robots.txt or bot-management settings block a provider's crawler, that provider cannot return your pages to the models that use it. Review your settings on purpose rather than by default. Our robots.txt guide covers the main user agents.
- Clean, fast pages travel further. Search results passed to a model are usually titles, URLs, snippets, and extracted text. Pages with clear headings and a direct answer near the top survive that compression best.
This publication runs on Cloudflare, and we will keep a close eye on which crawlers these providers send and how they identify themselves.
Free newsletter
The Sherpa Brief
One email a week on what changed in AI search, what it means, and what to do about it. Written for marketers, SEOs, and publishers.
Free. Unsubscribe in one click.
