Publishers who block Google-Extended expect Gemini to stay away from their pages. Facebook blocks it, and a Gemini line is showing up anyway.

Facebook's robots.txt disallows Google-Extended, the token Google says publishers can use to keep their content out of Gemini training and out of grounding in Gemini Apps. On November 5, SEO consultant Glenn Gabe posted Semrush screenshots that he says show visibility for Facebook's AI-written /fb-answers/ pages in AI Overviews, AI Mode and Gemini.

That is not proof that the Gemini app uses those pages. My read is that the first job is to find out what Semrush's Gemini line measures.

The story starts with a Peripheral article by search expert Saul Westbrook, published November 4.

Westbrook noticed in June that Meta had quietly indexed millions of AI-generated pages under facebook.com/fb-answers/. Each page is an AI summary of one topic, built from public Facebook posts. Semrush estimates that over 39 million of these pages rank in the US for at least one keyword, and that they have driven over 100 million visits worldwide. Those are estimates. Westbrook's table puts the monthly worldwide total at 1,032,527 visits in July and 40,590,906 in October.

Column chart of Semrush's estimated monthly worldwide visits to Facebook Answers pages, July to October 2026
Four months from a standing start, on Semrush's estimates.

He also wrote that Facebook had not commented as of November 3.

Gabe picked it up the next day. Search Engine Roundtable quotes him saying there are "6M+ urls indexed there and the directory ranks for millions of queries." Executive editor Barry Schwartz called it scaled content abuse in his headline and wrote, "I suspect this stuff won't be indexed too much longer, now that it is out there publicly."

Westbrook sees a way out for Facebook. "It is possible that they will avoid being penalised, as all content is sourced from their own first-party data," he wrote, and he notes that the pages survived the September spam update.

The spam argument is the loud one. The robots.txt question is quieter.

Westbrook wrote that "LLMs are blocked from crawling these pages." Facebook's robots.txt, read on November 6, matches that for most of the named AI crawlers. The groups for GPTBot, ClaudeBot, PerplexityBot, Amazonbot, Applebot-Extended and Google-Extended each disallow everything, and so does the catch-all group. The Googlebot and Bingbot groups do not mention /fb-answers/. Neither is barred from the pages. Permission is not proof of a crawl, and I could not open an fb-answers page to look at it.

The rules fit ChatGPT, Claude and Perplexity staying away. They do not obviously fit Gemini. Google's documentation says Google-Extended manages whether crawled content can be used for "grounding (providing content from the Google Search index to the model at prompt time to improve factuality and relevancy)" in Gemini Apps. The same page says it "does not impact a site's inclusion in Google Search." That makes AI Overviews and AI Mode citations no conflict, since they are Search features. Only the Gemini line in Gabe's screenshots is.

Gabe wrote, "And here is visibility across AIOs, AI Mode, and Gemini. Insane." I could not open his posts, and I cannot read the Semrush figures in them. They stay out of this story.

I see three explanations, and the screenshots cannot pick between them. Semrush's Gemini measure may not track the Gemini app at all. Gemini may surface a URL from the Search index without using the page's content as grounding. Or Google-Extended may not be stopping this use of the pages.

I will not pick one. Anyone who wants an answer needs to ask Semrush how its Gemini data is collected, and then run many logged-out Gemini prompts on topics where fb-answers pages rank, because one answer tells you little. Until then, the claim that Google-Extended has failed is as unsupported as the claim that it works.

Westbrook suggests brands post on Facebook to shape these summaries. If the major AI crawlers named in the file cannot read the pages, that work may reach Google's surfaces and not ChatGPT, Claude or Perplexity.

Free newsletter

The Sherpa Brief

One email a week on what changed in AI search, what it means, and what to do about it. Written for marketers, SEOs, and publishers.

Free. Unsubscribe in one click.