Your page answers the question cold, and ChatGPT cites someone else. Your first instinct is to rewrite the page.

A thread on r/GEO_optimization on November 21, 2026, says to check something simpler first: whether OpenAI's crawler ever fetched the page at all. The thread's sharper split: OAI-SearchBot crawls are the floor, and ChatGPT-User fetches are the signal, because only a real prompt pulls the page in.

The numbers behind it come from Resoneo's read of 1,200 ChatGPT answers. Pages the model opened were cited 74 percent of the time, against 7 percent for retrieved-but-unopened pages.

Page cache hides crawler visits from WordPress. WordPress-side crawler counts are a floor. My read is that the diagnostic order is still right: check the fetch before you touch the copy.

A developer who runs agentnumber10.com laid the whole thing out on r/GEO_optimization on November 21.

Jaded-Passenger-8072 started from the question he kept seeing: why ChatGPT is not citing us. His answer is that everyone skips the step before it: "A page that OpenAI's and Perplexity's crawlers have never fetched is a long shot for a citation, and some pages may never have been fetched at all." He disclosed in the post that agentnumber10.com sells an AI visibility tool.

He turned the check into a free WordPress plugin. It lists the pages no AI crawler has ever fetched. The directory page lists it at version 0.1.1 with no reviews yet.

It checks each claimed crawler visit against the operator's published IP list and labels it Verified, Spoofed or Unverified. Arrivals from AI answers are counted per page in the browser, up to 2,000 a day. Counting in the browser means cached pages count too.

Disclosure: my agency, ViewEngine, sells AI visibility work and builds its own tracking tools.

The crawler split he relies on is straight from OpenAI's crawler documentation. OAI-SearchBot and GPTBot are independent robots.txt tags.

OpenAI's crawler documentation showing the independent robots.txt tags paragraph
The independent-tags paragraph, as it reads on OpenAI's page. Screenshot: OpenAI Developers, Nov. 21, 2026

A site can allow the first for search while disallowing the second for training. Pages opted out of SearchBot will not appear in ChatGPT search answers, and OpenAI says robots.txt changes take about 24 hours to take effect.

The cache warning is on the plugin page too. When a page cache serves the request, WordPress never runs and no plugin can log that crawler visit. WordPress-side crawler counts are a floor.

A commenter called Heavivert pushed back first. "Crawl coverage is a floor, not a signal [...] we have pages OAI-SearchBot fetches weekly that never get cited, and one bare FAQ that got cited constantly because it was the only page answering a category question directly."

Then a commenter posting as aidanbuckley_aeo landed the split this piece is named for. "SearchBot is the index crawl [...] ChatGPT-User is a fetch triggered from inside a conversation, so when it shows up it's a much closer signal that a real prompt pulled your page in."

"pages SearchBot hits every week that never get a ChatGPT-User fetch or a citation are probably indexed fine and just not matching the searches chatgpt runs, so more crawl access won't fix them." Take that sentence from aidanbuckley_aeo as a hypothesis, not a diagnosis.

Hits in a log show a page was crawled, not that it is indexed, and zero User fetches do not say why the page was passed over. The comparison points at pages to investigate, not pages with a confirmed problem.

OpenAI's docs back the distinction. ChatGPT-User visits pages from user questions or Custom GPTs, is not automatic crawling, and is not used to determine Search appearance. Because the fetch starts with a user, robots.txt rules may not apply.

Opened beats retrieved

The closest independent support comes from Resoneo, reported in Search Engine Land. Its team captured 1,200 ChatGPT answers with a Chrome extension that records the raw data stream, tracing 88,000 search results across 26,900 pages.

Across that corpus, 61,332 URLs surfaced and 759 pages were opened, all in thinking mode. An opened page ended up cited 74 percent of the time. A page merely retrieved but never opened was cited 7 percent of the time.

Column chart: pages ChatGPT opened were cited 74 percent of the time, against 7 percent for retrieved-but-unopened pages
Every ChatGPT-User fetch in your logs is a page playing for the left bar.

That is an association between opens and citations, not proof that a weekly log comparison diagnoses any given page.

Resoneo attributes those opens to the ChatGPT-User robot, which only fetches when a user is behind the question. What the signal demonstrates is a user-triggered fetch, not a citation.

The study found the pages the model opened on its own carry no utm_source. The clicks show up in analytics. The reads never do. Resoneo reports that in instant mode the model worked from a 200-character snippet cut at indexing time rather than the live page.

The same split showed up on X the same day. Rios, posting as @Riosweb3, sorted bots into three buckets, with live agents on top. "Live agents fetch a page because someone just asked a question (ChatGPT-User, DuckAssistBot) [...] The third kind matters most. That's a real person's question landing on your site."

Not everyone buys the on-site framing. One related comment argued the game is decided off site, in the roundups and Reddit threads ChatGPT already cites. Crawl fixes alone will not move mentions.

I keep arguing the same thing from the other side: what matters is everything the AI can find about a brand across the web, and what it concludes after reading it.

A missing recommendation may need no on-site fix at all. The thread answers the narrower question of which of your pages to fix first, once you have decided the fix is on site.

The same split is in Perplexity's crawler documentation. PerplexityBot builds its search index and follows robots.txt, while Perplexity-User fetches on a user's question and generally ignores robots.txt.

My read is that Heavivert and aidanbuckley_aeo have the order right. Retrieval and citation are different events, and the Resoneo numbers show how far apart they can sit.

A page can be read and passed over this time and cited next time. The fetch belongs at the start of the diagnosis, not the end of it.

I track AI visibility every week across AI Overviews, AI Mode, ChatGPT, Perplexity and Gemini. The failure mode I keep coming back to is the rewrite before the diagnosis: changing the page before checking whether the crawler ever saw it.

Run the split before the rewrite. Verify the crawler against published IPs in a few weeks of server or CDN logs, and list the pages SearchBot has not touched in that window. Then compare steady SearchBot hits with zero ChatGPT-User fetches to shortlist pages worth investigating.

Citations are the diagnostic, not the prize. The prize is the recommendation and the brand mention, which is what sends the branded search. Start with the fetch, and the rest of the diagnosis gets easier.

Free newsletter

The Sherpa Brief

One email a week on what changed in AI search, what it means, and what to do about it. Written for marketers, SEOs, and publishers.

Free. Unsubscribe in one click.