AI Robots.txt Checker
Check whether GPTBot, ClaudeBot, PerplexityBot, and Google-Extended are allowed to read your Shopify storefront — and see live fetch status for key bots.
Why this matters
A single Disallow line can quietly stop ChatGPT, Claude, or Perplexity from ever reading your PDPs. This checker maps your robots.txt against the AI crawlers that matter for answer engines.
How it works
Fetch robots.txt
We retrieve your live robots.txt and parse every User-agent, Allow, and Disallow rule.
Match AI bots
We evaluate GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot, Google-Extended, and peers against path /.
Live bot fetch
We also request your homepage as GPTBot, ClaudeBot, and PerplexityBot to catch WAF/block pages robots.txt alone misses.
What the report covers
Allow vs block status for each major AI User-Agent used in AEO/GEO.
HTTP status when fetching as GPTBot, ClaudeBot, and PerplexityBot.
Whether robots.txt points crawlers at a Sitemap URL.
FAQ
Is the robots.txt checker free?
Yes. Enter a domain and get an on-page report — no account required.
Will blocking AI bots hurt ChatGPT visibility?
It can. If GPTBot or PerplexityBot cannot fetch your pages, those engines cannot cite or recommend your products from your own site.
Does this edit my robots.txt?
No. The checker is read-only. It reports what to change; you update Shopify (or your SEO app) yourself.
Which bots should Shopify brands allow?
Allow GPTBot / OAI-SearchBot for ChatGPT, ClaudeBot for Claude, PerplexityBot for Perplexity, and Google-Extended for Gemini-related surfaces — while keeping checkout/account paths disallowed under *.
Scan your Shopify store free
TopMention is the Shopify AEO/GEO platform — see whether ChatGPT, Claude, Perplexity and Gemini mention your brand, and who's winning your category.