AI Visibility Grader

Methodology

How we check

This is a deterministic audit. When you grade a site, we fetch its robots.txt (a small text file on your site that tells crawlers what they may read), homepage, sitemap and llms.txt live, then run the same fixed rules over what comes back. No AI model decides your score. The plain-English summary in the full report is written by Claude, but it never changes the score.

Where a published standard or vendor documentation exists, the rule comes from it: the Robots Exclusion Protocol, each AI company's crawler documentation, Google's search documentation, schema.org and the sitemaps protocol. Where none exists, the threshold is our own judgement, and we say so below.

How many points each check is worth, and the score bands, are our editorial choice. The 6 categories add up to 100 points. A pass earns a check's full points, a partial half, a fail none, and informational checks are reported but never scored. If your site serves the same content, it always gets the same score. Some rules mention status codes, the number a server sends back with each request: 200 means it loaded, 4xx means the request was refused or the page was not found, and 5xx means the server itself failed.

01

Crawler access

25 pts

  • Homepage loads for crawlers

    5 pts

    Pass if your homepage returns status 200 over https:// to our crawler, with no bot check or challenge page. Partial if it loads only when we retry as an ordinary web browser, or ends up on plain http:// rather than https://. Fail if it returns any other status or shows a bot check or challenge page.

    Source: HTTP semantics (RFC 9110) and the tool's own fetch

  • OAI-SearchBot access (ChatGPT search)

    2 pts

    Pass if your robots.txt permits OAI-SearchBot to read your homepage, or there is no robots.txt (status 404). Fail if your robots.txt blocks OAI-SearchBot from your homepage. Partial if your robots.txt does not respond, returns a server error (5xx) or returns too many requests (429), which crawlers such as Googlebot treat as "stay out" until it recovers. Also partial if it refuses our request (401 or 403), because we then cannot tell which crawlers it permits. Other 4xx statuses count as no restrictions.

    Source: RFC 9309 (Robots Exclusion Protocol) and Google's robots.txt specification; crawler name from OpenAI crawler documentation

  • ChatGPT-User access (ChatGPT on request)

    2 pts

    Pass if your robots.txt permits ChatGPT-User to read your homepage, or there is no robots.txt (status 404). Fail if your robots.txt blocks ChatGPT-User from your homepage. Partial if your robots.txt does not respond, returns a server error (5xx) or returns too many requests (429), which crawlers such as Googlebot treat as "stay out" until it recovers. Also partial if it refuses our request (401 or 403), because we then cannot tell which crawlers it permits. Other 4xx statuses count as no restrictions.

    Source: RFC 9309 (Robots Exclusion Protocol) and Google's robots.txt specification; crawler name from OpenAI crawler documentation

  • Claude-SearchBot access (Claude search)

    2 pts

    Pass if your robots.txt permits Claude-SearchBot to read your homepage, or there is no robots.txt (status 404). Fail if your robots.txt blocks Claude-SearchBot from your homepage. Partial if your robots.txt does not respond, returns a server error (5xx) or returns too many requests (429), which crawlers such as Googlebot treat as "stay out" until it recovers. Also partial if it refuses our request (401 or 403), because we then cannot tell which crawlers it permits. Other 4xx statuses count as no restrictions.

    Source: RFC 9309 (Robots Exclusion Protocol) and Google's robots.txt specification; crawler name from Anthropic crawler documentation

  • Claude-User access (Claude on request)

    2 pts

    Pass if your robots.txt permits Claude-User to read your homepage, or there is no robots.txt (status 404). Fail if your robots.txt blocks Claude-User from your homepage. Partial if your robots.txt does not respond, returns a server error (5xx) or returns too many requests (429), which crawlers such as Googlebot treat as "stay out" until it recovers. Also partial if it refuses our request (401 or 403), because we then cannot tell which crawlers it permits. Other 4xx statuses count as no restrictions.

    Source: RFC 9309 (Robots Exclusion Protocol) and Google's robots.txt specification; crawler name from Anthropic crawler documentation

  • PerplexityBot access (Perplexity search)

    2 pts

    Pass if your robots.txt permits PerplexityBot to read your homepage, or there is no robots.txt (status 404). Fail if your robots.txt blocks PerplexityBot from your homepage. Partial if your robots.txt does not respond, returns a server error (5xx) or returns too many requests (429), which crawlers such as Googlebot treat as "stay out" until it recovers. Also partial if it refuses our request (401 or 403), because we then cannot tell which crawlers it permits. Other 4xx statuses count as no restrictions.

    Source: RFC 9309 (Robots Exclusion Protocol) and Google's robots.txt specification; crawler name from Perplexity crawler documentation

  • Googlebot access (Google Search and AI Overviews)

    2 pts

    Pass if your robots.txt permits Googlebot to read your homepage, or there is no robots.txt (status 404). Fail if your robots.txt blocks Googlebot from your homepage. Partial if your robots.txt does not respond, returns a server error (5xx) or returns too many requests (429), which crawlers such as Googlebot treat as "stay out" until it recovers. Also partial if it refuses our request (401 or 403), because we then cannot tell which crawlers it permits. Other 4xx statuses count as no restrictions.

    Source: RFC 9309 (Robots Exclusion Protocol) and Google's robots.txt specification; crawler name from Google crawler documentation

  • Bingbot access (Bing and Copilot)

    2 pts

    Pass if your robots.txt permits Bingbot to read your homepage, or there is no robots.txt (status 404). Fail if your robots.txt blocks Bingbot from your homepage. Partial if your robots.txt does not respond, returns a server error (5xx) or returns too many requests (429), which crawlers such as Googlebot treat as "stay out" until it recovers. Also partial if it refuses our request (401 or 403), because we then cannot tell which crawlers it permits. Other 4xx statuses count as no restrictions.

    Source: RFC 9309 (Robots Exclusion Protocol) and Google's robots.txt specification; crawler name from Microsoft crawler documentation

  • GPTBot access (OpenAI model training)

    1 pt

    Pass if your robots.txt permits GPTBot to read your homepage, or there is no robots.txt (status 404). Fail if your robots.txt blocks GPTBot from your homepage. Partial if your robots.txt does not respond, returns a server error (5xx) or returns too many requests (429), which crawlers such as Googlebot treat as "stay out" until it recovers. Also partial if it refuses our request (401 or 403), because we then cannot tell which crawlers it permits. Other 4xx statuses count as no restrictions.

    Source: RFC 9309 (Robots Exclusion Protocol) and Google's robots.txt specification; crawler name from OpenAI crawler documentation

  • ClaudeBot access (Anthropic model training)

    1 pt

    Pass if your robots.txt permits ClaudeBot to read your homepage, or there is no robots.txt (status 404). Fail if your robots.txt blocks ClaudeBot from your homepage. Partial if your robots.txt does not respond, returns a server error (5xx) or returns too many requests (429), which crawlers such as Googlebot treat as "stay out" until it recovers. Also partial if it refuses our request (401 or 403), because we then cannot tell which crawlers it permits. Other 4xx statuses count as no restrictions.

    Source: RFC 9309 (Robots Exclusion Protocol) and Google's robots.txt specification; crawler name from Anthropic crawler documentation

  • Homepage allows indexing and quoting

    4 pts

    Fail if your homepage's robots meta tag or X-Robots-Tag header (hidden instructions sent with the page) contains noindex, none, nosnippet or max-snippet:0. Partial if we could not read your homepage, so the meta tag could not be checked. Pass otherwise.

    Source: Google's robots meta tag documentation (noindex, nosnippet, max-snippet apply to AI Overviews and AI Mode)

  • Google-Extended access (Gemini training and grounding)

    Informational

    Informational, never scored. Reports whether your robots.txt permits Google-Extended, using the same robots.txt rules as the scored crawlers. Google-Extended is not a separate crawler: it controls whether Google may use your content to train Gemini models and ground Gemini's answers, and it does not affect Google Search or AI Overviews.

    Source: RFC 9309 (Robots Exclusion Protocol) and Google's robots.txt specification; crawler name from Google crawler documentation

  • Perplexity-User access (Perplexity on request)

    Informational

    Informational, never scored. Reports whether your robots.txt permits Perplexity-User, using the same robots.txt rules as the scored crawlers. Because a person asked for the page, Perplexity-User generally ignores robots.txt, so a rule here has little effect.

    Source: RFC 9309 (Robots Exclusion Protocol) and Google's robots.txt specification; crawler name from Perplexity crawler documentation

02

Content readability

20 pts

  • Amount of readable text

    5 pts

    Pass at 300 or more words of readable text (not counting scripts, navigation, header or footer). Partial at 100 to 299 words. Fail below 100.

    Source: Editorial thresholds chosen by this tool, informed by general search-quality guidance

  • Content readable without JavaScript

    5 pts

    Fail if the page your server sends has fewer than 30 words of readable text and an app container (an element with the id or class root, app, __next, __nuxt or app-root) holding fewer than 30 words, the pattern of a page built by JavaScript. Pass otherwise.

    Source: Published crawler analyses (for example Vercel's 2024 study of AI crawler behaviour) show GPTBot, OAI-SearchBot, ClaudeBot and PerplexityBot fetch raw HTML without executing JavaScript; Google documents that Googlebot renders JavaScript.

  • One main heading

    3 pts

    Pass with exactly one main heading (H1). Partial with more than one. Fail with none.

    Source: Editorial thresholds chosen by this tool, informed by general search-quality guidance

  • Subheadings that break up the page

    3 pts

    Pass with 2 or more subheadings (H2 or H3) outside the page's navigation, header and footer. Fail with fewer than 2.

    Source: Editorial thresholds chosen by this tool, informed by general search-quality guidance

  • Text compared with code

    2 pts

    Pass if readable text is more than 10% of the homepage's HTML by length. Partial at more than 5% up to 10%. Fail at 5% or less.

    Source: Editorial thresholds chosen by this tool, informed by general search-quality guidance

  • Page language declared

    2 pts

    Pass if the opening html tag declares a language (lang attribute). Fail otherwise.

    Source: Editorial thresholds chosen by this tool, informed by general search-quality guidance

03

Structured data

20 pts

  • Structured data reads without errors

    6 pts

    Pass if every structured data (JSON-LD) block parses. Partial if some parse and some have errors. Fail if there are none, or none parse.

    Source: schema.org vocabulary and Google's structured data guidelines

  • Organization structured data

    6 pts

    Pass if your structured data includes an Organization (or a more specific type such as LocalBusiness) with both a name and a url. Partial if it is missing either. Fail if there is none.

    Source: schema.org vocabulary and Google's structured data guidelines

  • WebSite or WebPage structured data

    4 pts

    Pass if your structured data includes a WebSite or WebPage type. Fail otherwise.

    Source: schema.org vocabulary and Google's structured data guidelines

  • FAQ, breadcrumb, article or product structured data

    4 pts

    Pass if your structured data includes FAQPage, BreadcrumbList, an Article type (including BlogPosting) or Product. Partial if it has none of these but at least one structured data block parses. Fail otherwise.

    Source: schema.org vocabulary and Google's structured data guidelines

04

Answer-readiness

15 pts

  • Headings phrased as questions

    3 pts

    Pass if any heading ends with a question mark or starts with who, what, when, where, why, how, can, does, is or are, or if a heading mentions FAQ or frequently asked questions. Fail otherwise.

    Source: Editorial thresholds chosen by this tool, informed by general search-quality guidance

  • FAQ markup

    3 pts

    Pass if your structured data includes FAQPage. Fail otherwise.

    Source: schema.org vocabulary and Google's structured data guidelines

  • Lists and tables

    3 pts

    Pass with at least one bulleted list, numbered list or table in the main content (the main element, or the page outside its navigation, header and footer if there is none). Fail with none.

    Source: Editorial thresholds chosen by this tool, informed by general search-quality guidance

  • Clear opening paragraph

    3 pts

    Pass if the first paragraph after the main heading (H1) is 40 to 160 words. Partial if it is shorter or longer. Fail if there is no main heading or no paragraph after it.

    Source: Editorial thresholds chosen by this tool, informed by general search-quality guidance

  • Meta description

    3 pts

    Pass if the meta description is 50 to 160 characters. Partial if it is shorter or longer. Fail if there is none.

    Source: Editorial thresholds chosen by this tool, informed by general search-quality guidance

05

Entity & trust signals

10 pts

  • Page title names your business

    2 pts

    Pass if the page title is 10 to 70 characters and includes your business name, ignoring case, spaces and punctuation. Your business name is the Organization name from your structured data or the main part of your domain name (acme for acme.co.uk). Partial if it has a title that misses either. Fail if there is no title.

    Source: HTML and Open Graph conventions; schema.org sameAs

  • Canonical address points to your own site

    2 pts

    Pass if the canonical tag (the address you declare as the official one for the page) points to an address on your own site (the same host as your homepage). Fail if it is missing, is not a valid web address, or points to another host.

    Source: HTML and Open Graph conventions; schema.org sameAs

  • Open Graph title and description

    2 pts

    Pass with both og:title and og:description (Open Graph tags, the title and description used when a page is shared). Partial with one of them. Fail with neither.

    Source: HTML and Open Graph conventions; schema.org sameAs

  • Links to About and Contact pages

    2 pts

    Pass if your homepage links to a page on your own site whose address or link text contains "about" or "contact". Fail otherwise.

    Source: HTML and Open Graph conventions; schema.org sameAs

  • Links to your official profiles (sameAs)

    2 pts

    Pass if your Organization structured data has a sameAs list (links to your official profiles) with at least one entry. Fail otherwise.

    Source: HTML and Open Graph conventions; schema.org sameAs

06

Technical basics

10 pts

  • XML sitemap

    5 pts

    We look for the first Sitemap line in your robots.txt, or /sitemap.xml if there is none. Pass if it returns status 200 and is not a web page. Fail if it is missing, does not respond, returns another status, or returns a web page instead of a sitemap.

    Source: sitemaps.org protocol

  • llms.txt file

    2 pts

    Pass if /llms.txt returns status 200 and is not a web page. Fail otherwise.

    Source: llms.txt proposal (emerging convention, not a standard)

  • Redirect chain

    3 pts

    Pass with 2 or fewer redirects before your homepage loads. Partial with 3. Fail with 4 or more.

    Source: HTTP semantics (RFC 9110) and the tool's own fetch

  • Homepage response time

    Informational

    Informational, never scored. Reports how long your homepage took to respond to our server: under 1s, 1 to 3s, or over 3s. A single measurement is not reliable enough to score.

    Source: HTTP semantics (RFC 9110) and the tool's own fetch

What we cannot see

  • robots.txt is a request, not a lock. Your robots.txt may permit a crawler while your firewall or CDN (the service that delivers your site, such as Cloudflare) still blocks it. We cannot see those rules, so we report what your robots.txt permits.
  • We grade your homepage only. Any path you enter is dropped, and we do not follow links.
  • We do not run JavaScript, by design. Most AI crawlers, including GPTBot, OAI-SearchBot, ClaudeBot and PerplexityBot, read the page exactly as your server sends it, so that is what we check. Googlebot does run JavaScript, so Google may see content that we do not.
  • If your site refuses both our crawler and our retry as an ordinary browser (any status other than 200, or a bot check page), we cannot read your homepage. Content readability, Structured data, Answer-readiness and Entity & trust signals are then marked unassessed and score nothing, while the Crawler access and Technical basics checks still run.

Last reviewed: 17 October 2026

Grade a site