--- title: "AI search readiness: can ChatGPT read every client's site? · MetricDeck" description: "Check every client's site daily for what keeps it out of ChatGPT, Claude, Perplexity and Google's AI Overviews, and hear when an AI crawler gets blocked." url: https://metricdeck.app/features/ai-search-readiness updated: 2026-10-05 --- AI search readiness # Know the day ChatGPT can't read a client's site Check every client's site each night for what keeps ChatGPT, Claude and Perplexity out, and hear when that changes. [Start free](https://metricdeck.app/signup) [See the live demo](https://metricdeck.app/demo) * Free for one client * No card * Read-only access Trailhead Coffee · Site health · AI search readiness Sample data The numbers behind it robots.txt for OAI-SearchBot Disallow: / (User-agent: oai-searchbot) robots.txt for ChatGPT-User Disallow: / (User-agent: chatgpt-user) robots.txt for Claude-SearchBot Disallow: / (User-agent: claude-searchbot) robots.txt for Claude-User Disallow: / (User-agent: claude-user) Score 75 / 100 Needs improvement trailheadcoffee.example · checked today | Assistant | Can quote it in answers | Can learn from it | | ---------------------------- | --------------------------------------- | ------------------- | | ChatGPT | Blocked (OAI-SearchBot, ChatGPT-User) | Blocked (GPTBot) | | Claude | Blocked (Claude-SearchBot, Claude-User) | Blocked (ClaudeBot) | | Perplexity | Allowed | No crawler for this | | Google AI Overviews & Gemini | Allowed | Allowed | | Microsoft Copilot | Allowed | No crawler for this | | Apple Intelligence | Allowed | Allowed | | Meta AI | No crawler for this | Allowed | | Common Crawl | No crawler for this | Allowed | * fail AI assistants can read the page17/30 robots.txt blocks ChatGPT, Claude (OAI-SearchBot, ChatGPT-User, Claude-SearchBot, Claude-User), so they can't quote or link this page. **How to fix:** Remove the Disallow rules for OAI-SearchBot, ChatGPT-User, Claude-SearchBot, Claude-User in robots.txt. * pass Page text is in the HTML20/20 540 words readable without running JavaScript. * pass Allowed in search and AI answers15/15 No noindex or nosnippet on the page. * warn Structured data (schema.org)10/15 Found CafeOrCoffeeShop, PostalAddress. Nothing says who's behind it (Organization, LocalBusiness, Product…). **How to fix:** Add Organization or LocalBusiness JSON-LD with the business's name, logo and contact details. * pass Title and description5/5 “Trailhead Coffee – roastery and café in Baner, Pune” * pass Sitemap5/5 Crawlers can find every page from the sitemap. * warn llms.txt0/5 No /llms.txt: an optional Markdown summary of the site for AI tools. **How to fix:** Publish /llms.txt: the site's name, what it offers, and links to its key pages in Markdown. * warn AI models can learn from the page3/5 Blocked for ChatGPT, Claude. A fair choice, but those models learn less about the business, so they mention it less. Blocking training crawlers is a fair choice and costs little here; blocking the answer crawlers keeps the page out of AI answers altogether. Trailhead Coffee is one of the demo agency's made-up clients. The alerts are what MetricDeck's rules raise on its latest check, and the report below is the same one the app and the free checker show. Every page the demo agency watches, checked today * northwinddental.example Northwind Dental AI assistants can read and quote it 95/100 * lumenskincare.example Lumen Skincare AI assistants can read and quote it 100/100 * trailheadcoffee.example Trailhead Coffee robots.txt blocks ChatGPT, Claude 75/100 * trailheadcoffee.example/shop Trailhead Coffee robots.txt blocks ChatGPT, Claude 80/100 Sample data. Want to check one page now? [Try the free AI search checker](https://metricdeck.app/tools/ai-search-checker). ## The check, before and after Making sure one client page can be read and quoted by AI assistants. Time you get back 2 h 20 min Each time you check, if you look after 10 clients. For one client By hand 15 min With MetricDeck 1 min Our estimate for one page per client, checked by hand. Most people check far less often than every plugin update or deploy, which is how blocks go unnoticed for months. By hand 15 min 1. 01 Open robots.txt and read it for each AI crawler: GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot and the rest 5 min 2. 02 View the page source to see what's there before JavaScript runs 3 min 3. 03 Look for noindex in the page and its headers 2 min 4. 04 Run the structured data through a validator 3 min 5. 05 Check for a sitemap and llms.txt 2 min With MetricDeck 1 min 1. 01 Nothing to run: every page is checked each night, and an alert says when an answer crawler is blocked Automatic 2. 02 Read the score and the fixes on Site health when you want the detail 1 min ## What you get * ### A score for every page Eight checks worth 100 points, each with what it found and how to fix it. * ### Assistant by assistant Whether ChatGPT, Claude, Perplexity, Google's AI Overviews, Copilot and Apple may quote the page and learn from it, with the robots.txt rule that blocks one. * ### An alert when it breaks When an answer crawler is blocked (critical if it's Googlebot), robots.txt answers with an error, or the page says noindex. Blocking training crawlers is your call and never alerts. * ### What crawlers really see The words in the HTML before JavaScript runs, the title and description, and the structured data types on the page. * ### Every page you watch The website plus the extra pages you add: 2 on Starter, 5 on Pro and 10 on Agency. * ### In the summary and the report The AI summary reads it with everything else, and the client report has an AI search readiness section for each page. ## Why you can trust the result * No AI in the check: it's rules applied to what a crawler gets, so the same page always gives the same answer. * robots.txt is read the way RFC 9309 says, for each crawler, on the page's own path. * Training crawlers are your choice: blocking them costs a few points and never raises an alert. * It's the same check as the free AI search checker and the Site health tab, so they always agree. FAQ ## Questions How is this different from the free AI search checker? It's the same check and the same report. The free checker looks at one page, once, when you ask. MetricDeck checks every page you watch for every client each night, tells you when an assistant gets shut out, and puts the result in the client's report. Does blocking GPTBot keep a site out of ChatGPT? No. GPTBot only collects pages to train OpenAI's models. ChatGPT's search, and the pages it opens while answering, use OAI-SearchBot and ChatGPT-User; those are the ones that decide whether a site can be quoted. Why would a site block AI assistants without meaning to? Usually a "block AI" list copied into robots.txt that names the answer crawlers along with the training ones, a rule left over from a staging site, or a security plugin's setting. Pages that build their text in the browser are the other common gap: the crawlers behind ChatGPT, Claude and Perplexity don't run JavaScript. Does it catch a firewall that turns AI crawlers away? Not yet. MetricDeck reads robots.txt and fetches the page as its own checker, so a firewall rule that refuses only AI crawlers won't show up. robots.txt, the page's text, noindex and the rest are what it checks. How often is each page checked? With every full sync: each night, and whenever you press Sync now. Not hourly, since robots.txt and a page's content don't change by the hour. Which pages are checked? The project's website, plus the extra pages you add on its Site health tab: 2 on Starter, 5 on Pro and 10 on Agency. ## More of what MetricDeck does * [Attention page Every client's Search Console, Analytics, Ads and website checked overnight: open alerts worst first, and each client's week against the one before. ](https://metricdeck.app/features/attention) * [Alerts Alerts when ad spend jumps, conversions stop, traffic or rankings drop, tracking breaks or a site goes down, each with its numbers and an AI explanation. ](https://metricdeck.app/features/alerts) * [AI summary A summary of each client after every sync: what needs fixing, what went well and what to do next. Code works out every number; the AI only explains them. ](https://metricdeck.app/features/ai-summary) * [Client reports Pick the sections, edit the AI summary, and send a PDF or a link with a password: Search Console, Analytics, Ads and site health in one client report. ](https://metricdeck.app/features/reports) * [Search Console reporting Clicks, impressions, CTR and position for every client, the pages and queries that lost clicks, and an alert when organic traffic drops. ](https://metricdeck.app/features/search-console) * [Google Analytics reporting Sessions and key events by channel, landing page, device and country, visits from ChatGPT and Perplexity counted apart, and alerts on drops. ](https://metricdeck.app/features/analytics) * [Google Ads wasted spend See the Google Ads search terms that spend without converting, the ones that don't match what you sell, and copy a negative keyword list. Read-only. ](https://metricdeck.app/features/google-ads) * [ChatGPT ads reporting Every client's ChatGPT ads from the OpenAI Ads API: spend, clicks, conversions, lifetime budgets running out and rejected ads, beside Google Ads. ](https://metricdeck.app/features/chatgpt-ads) * [Site health monitoring Hourly uptime checks confirmed before a site is called down, SSL and domain expiry, and mobile and desktop speed for every client's site, with alerts. ](https://metricdeck.app/features/site-health) ## Watch every client's site for AI search Add a project and its website is checked straight away, AI search readiness included. Free for one site. [Start free](https://metricdeck.app/signup) [See the live demo](https://metricdeck.app/demo) Free for one client · No card · Read-only access