Skip to content
PricingPlans from $15 a month billed yearly: ~20% less than monthly
MetricDeck
Log inStart free

Free tool · no sign-up

Can ChatGPT read your website?

Check whether ChatGPT, Claude, Perplexity, Google's AI Overviews and Copilot can read, understand and quote any page, the way their crawlers actually read it.

What it checks

Eight checks, 100 points

Each failing one comes with how to fix it.

  • AI assistants can read the page

    30 points

    robots.txt, read for each assistant's answer crawler on this page's own path. A blocked crawler can't quote or link the page.

  • Page text is in the HTML

    20 points

    The words in the HTML your server sends. Most AI crawlers don't run JavaScript, so text built in the browser is invisible to them.

  • Allowed in search and AI answers

    15 points

    No noindex or nosnippet in the page or its headers. Either one keeps the page out of Google's AI Overviews and Copilot.

  • Structured data (schema.org)

    15 points

    JSON-LD structured data that says what the page is and who's behind it: Organization, LocalBusiness, Product, Service.

  • Title and description

    5 points

    A title and a one-line description: the page's summary for any reader, person or model.

  • Sitemap

    5 points

    A sitemap listed in robots.txt or at /sitemap.xml, so crawlers can find the rest of the site.

  • llms.txt

    5 points

    An optional /llms.txt: a plain summary of the site and its key pages for AI tools.

  • AI models can learn from the page

    5 points

    Whether models may learn from the page. Blocking this is a fair choice, and costs only a few points.

Crawlers

Which crawler belongs to which assistant

Each assistant uses one set of crawlers to fetch pages for its answers, and often another to collect pages for training. robots.txt can treat them differently.

AssistantReads pages for answersCollects pages for training
ChatGPTOAI-SearchBot, ChatGPT-UserGPTBot
ClaudeClaude-SearchBot, Claude-UserClaudeBot
PerplexityPerplexityBot, Perplexity-User–
Google AI Overviews & GeminiGooglebotGoogle-Extended
Microsoft Copilotbingbot–
Apple IntelligenceApplebotApplebot-Extended
Meta AI–meta-externalagent
Common Crawl–CCBot

A robots.txt that lets them in

Anything not disallowed is allowed, so most sites need no change. If yours blocks crawlers, naming the answer crawlers like this lets assistants quote you while your other rules stay as they are.

# Let AI assistants find, read and quote the site
User-agent: OAI-SearchBot
User-agent: ChatGPT-User
User-agent: Claude-SearchBot
User-agent: Claude-User
User-agent: PerplexityBot
User-agent: Perplexity-User
User-agent: Googlebot
User-agent: bingbot
User-agent: Applebot
Allow: /

Sitemap: https://yourwebsite.com/sitemap.xml

FAQ

Questions

Does blocking GPTBot keep my site out of ChatGPT?
No. GPTBot collects pages to train OpenAI's models. ChatGPT's search, and the pages it opens while answering, use OAI-SearchBot and ChatGPT-User. Blocking GPTBot only opts you out of training; blocking OAI-SearchBot keeps you out of ChatGPT's answers.
What's the difference between Googlebot and Google-Extended?
Google's AI Overviews and AI Mode use what Googlebot crawls for Search, so blocking Googlebot takes you out of Google altogether. Google-Extended only controls whether your pages are used for Gemini models; it doesn't change Search or AI Overviews.
Why does it matter whether my page needs JavaScript?
The crawlers behind ChatGPT, Claude and Perplexity read the HTML your server sends and don't run JavaScript. A page that builds its text in the browser looks almost empty to them, so there's nothing to quote. Server-side rendering or pre-rendering fixes it.
What is llms.txt?
A proposed standard (llmstxt.org): a plain-text Markdown file at the root of your site that says what the site is and links its key pages, so an AI tool can understand it in one fetch. It's optional and not every assistant reads it yet, so it's worth a few points here.
Does this checker use AI?
No. Every result comes from rules applied to what a crawler would get: your robots.txt (read the way RFC 9309 says), the page's HTML, the sitemap and llms.txt. Run it twice and you get the same answer.
Is it free?
Yes, with no sign-up. To have a site checked every day, with uptime, speed, SSL and domain expiry too, and be told when something breaks, MetricDeck's free plan covers one site.

Every client's site, checked every day

MetricDeck runs this check on every site you look after, with uptime, speed, SSL and domain expiry, beside Search Console, Analytics and Google Ads, and tells you which client needs you today.

Start freeSee the live demo

Free for one client · No card · Read-only access