Is AI crawling my site?

The AI Search Console answers the first question: can the well-known AI bots reach your site at all?

Open the console for your site. We send a request that uses each bot’s User-Agent string — GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot, and the others we list. A 200 means that request got through. A 403, a challenge page, or a timeout means something in front of your site blocked our test.

That test is a simulated crawl from Knowatoa servers. We are not OpenAI, Anthropic, or Perplexity. We cannot send the request from those companies’ IP addresses. Bot-management tools that verify a crawler by its source network (Cloudflare Bot Fight Mode is the usual case) may block the test and still let the real bot through.

Use the console as a first check, not as the last word:

Every bot fails — a firewall or bot-fight layer is probably challenging all non-browser traffic, including our test. See Bot Failures & Investigations.
One or two bots fail — look at robots.txt and any rule aimed at that User-Agent.
The console is green — the path is open to a spoofed User-Agent. That is useful. It still does not tell you which pages the real bots are reading.

To see which pages real crawlers and agents request, and to tell a crawl from a live retrieval, use your CDN or server logs. Start with Which pages are AI bots crawling?.