--- title: "Free AI search checker: can ChatGPT read your website? · MetricDeck" description: "See if ChatGPT, Claude, Perplexity, Google's AI Overviews and Copilot can read and quote your website: robots.txt, page text, schema, llms.txt. Free." url: https://metricdeck.app/tools/ai-search-checker updated: 2026-10-04 --- Free tool · no sign-up # Can ChatGPT read your website? Check whether ChatGPT, Claude, Perplexity, Google's AI Overviews and Copilot can read, understand and quote any page, the way their crawlers actually read it. What it checks ## Eight checks, 100 points Each failing one comes with how to fix it. * ### AI assistants can read the page 30 points robots.txt, read for each assistant's answer crawler on this page's own path. A blocked crawler can't quote or link the page. * ### Page text is in the HTML 20 points The words in the HTML your server sends. Most AI crawlers don't run JavaScript, so text built in the browser is invisible to them. * ### Allowed in search and AI answers 15 points No noindex or nosnippet in the page or its headers. Either one keeps the page out of Google's AI Overviews and Copilot. * ### Structured data (schema.org) 15 points JSON-LD structured data that says what the page is and who's behind it: Organization, LocalBusiness, Product, Service. * ### Title and description 5 points A title and a one-line description: the page's summary for any reader, person or model. * ### Sitemap 5 points A sitemap listed in robots.txt or at /sitemap.xml, so crawlers can find the rest of the site. * ### llms.txt 5 points An optional /llms.txt: a plain summary of the site and its key pages for AI tools. * ### AI models can learn from the page 5 points Whether models may learn from the page. Blocking this is a fair choice, and costs only a few points. Crawlers ## Which crawler belongs to which assistant Each assistant uses one set of crawlers to fetch pages for its answers, and often another to collect pages for training. robots.txt can treat them differently. | Assistant | Reads pages for answers | Collects pages for training | | ---------------------------- | ------------------------------ | --------------------------- | | ChatGPT | OAI-SearchBot, ChatGPT-User | GPTBot | | Claude | Claude-SearchBot, Claude-User | ClaudeBot | | Perplexity | PerplexityBot, Perplexity-User | – | | Google AI Overviews & Gemini | Googlebot | Google-Extended | | Microsoft Copilot | bingbot | – | | Apple Intelligence | Applebot | Applebot-Extended | | Meta AI | – | meta-externalagent | | Common Crawl | – | CCBot | ### A robots.txt that lets them in Anything not disallowed is allowed, so most sites need no change. If yours blocks crawlers, naming the answer crawlers like this lets assistants quote you while your other rules stay as they are. # Let AI assistants find, read and quote the site User-agent: OAI-SearchBot User-agent: ChatGPT-User User-agent: Claude-SearchBot User-agent: Claude-User User-agent: PerplexityBot User-agent: Perplexity-User User-agent: Googlebot User-agent: bingbot User-agent: Applebot Allow: / Sitemap: https://yourwebsite.com/sitemap.xml FAQ ## Questions Does blocking GPTBot keep my site out of ChatGPT? No. GPTBot collects pages to train OpenAI's models. ChatGPT's search, and the pages it opens while answering, use OAI-SearchBot and ChatGPT-User. Blocking GPTBot only opts you out of training; blocking OAI-SearchBot keeps you out of ChatGPT's answers. What's the difference between Googlebot and Google-Extended? Google's AI Overviews and AI Mode use what Googlebot crawls for Search, so blocking Googlebot takes you out of Google altogether. Google-Extended only controls whether your pages are used for Gemini models; it doesn't change Search or AI Overviews. Why does it matter whether my page needs JavaScript? The crawlers behind ChatGPT, Claude and Perplexity read the HTML your server sends and don't run JavaScript. A page that builds its text in the browser looks almost empty to them, so there's nothing to quote. Server-side rendering or pre-rendering fixes it. What is llms.txt? A proposed standard (llmstxt.org): a plain-text Markdown file at the root of your site that says what the site is and links its key pages, so an AI tool can understand it in one fetch. It's optional and not every assistant reads it yet, so it's worth a few points here. Does this checker use AI? No. Every result comes from rules applied to what a crawler would get: your robots.txt (read the way RFC 9309 says), the page's HTML, the sitemap and llms.txt. Run it twice and you get the same answer. Is it free? Yes, with no sign-up. To have a site checked every day, with uptime, speed, SSL and domain expiry too, and be told when something breaks, MetricDeck's free plan covers one site. ## Every client's site, checked every day MetricDeck runs this check on every site you look after, with uptime, speed, SSL and domain expiry, beside Search Console, Analytics and Google Ads, and tells you which client needs you today. [Start free](https://metricdeck.app/signup) [See the live demo](https://metricdeck.app/demo) Free for one client · No card · Read-only access