siteTR Test

Discoverability

Can search engines — and AI systems — find, crawl and understand your pages at all?

0.20 of the composite score

Why does this matter — for rankings, traffic and trust?

Discoverability is the technical floor of every SEO result: before keywords, backlinks or content quality can matter, crawlers must be able to reach your pages, read them, and understand what each one is. A site with perfect content and a broken robots.txt, a missing sitemap, or duplicate titles across every page will simply not compete.

It is also your click-through-rate lever. The title tag and meta description are what Google actually renders in the result page — a missing, duplicated or oversized title quietly taxes every impression you earn. Canonical URLs and clean redirects decide which URL collects the ranking signals; without them you split your own equity across duplicates.

Since 2023 there is a second audience: AI answer engines (ChatGPT, Perplexity, AI Overviews) read robots.txt and sitemaps explicitly when they decide what to cite. Publishers who block everything lost a traffic channel overnight — discoverability now covers being reachable by machines that summarize, not just machines that index.

Every signal in this division — exactly what the engine checks

title tagw8Give every page a unique 30–60 character title.
XML sitemapw7Declare a sitemap in robots.txt and keep it reachable.
robots.txtw6Publish a robots.txt that references your sitemap.
HTTPS servingw6Serve the site over HTTPS and redirect HTTP permanently.
meta descriptionw5Add a 70–160 character meta description.
homepage responsew5Serve a plain 2xx HTML homepage to crawlers — check bot filtering, WAF rules and rewrites.
canonical URLw4Declare a self-referencing <link rel="canonical"> on every page.
Open Graph tagsw4Add og:title, og:description and og:image for link previews.
indexing directivesw4Remove the noindex meta tag or X-Robots-Tag header if this page should be indexed.
text/HTML ratiow4More of your bytes should be readable text — trim wrapper markup and inline bloat.
canonical qualityw3Keep exactly one canonical per page — absolute https:// and self-referencing unless you syndicate deliberately.
redirect chainw3Link and redirect straight to the final HTTPS URL — every extra hop costs latency and crawl budget.
HTTPS redirectw3301-redirect http:// requests to the same https:// URL.
DOM sizew3Flatten the DOM — huge element trees slow parsing, styling and LCP on phones.
Twitter/X cardw2Add <meta name="twitter:card" content="summary_large_image"> plus title/description/image.
language attributew2Set <html lang="en"> (or the page language) so browsers and engines pick the right locale.
faviconw1Publish a favicon (e.g. /favicon.svg) and link it with <link rel="icon">.

How to fix it

What commonly causes failures here

Best-practice setup

The three artifacts below cover most of this axis: a robots.txt that allows search and AI crawlers while keeping private areas out, a sitemap that always reflects reality, and a complete <head> on every page. Copy them and adjust.

robots.txt — allow search + AI crawlers, block private paths
User-agent: *
Disallow: /admin/
Disallow: /cart
Disallow: /*?sort=

# AI answer engines — decide deliberately, not by accident:
User-agent: GPTBot
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: Google-Extended
Allow: /

Sitemap: https://example.com/sitemap.xml
sitemap.xml — keep it honest, regenerate on deploy
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://example.com/</loc>
    <lastmod>2026-10-01</lastmod>
    <changefreq>weekly</changefreq>
    <priority>1.0</priority>
  </url>
  <url>
    <loc>https://example.com/pricing</loc>
    <lastmod>2026-09-28</lastmod>
    <priority>0.8</priority>
  </url>
</urlset>
HTML <head> — one unique block per page
<head>
  <title>Warehouse Automation Software | Acme Robotics</title>            <!-- 30–60 chars, unique -->
  <meta name="description" content="Cut picking time 40% with Acme's
    warehouse automation. Deployed in 200+ fulfillment centers.">        <!-- 70–160 chars -->
  <link rel="canonical" href="https://example.com/warehouse-automation">
  <meta property="og:title" content="Warehouse Automation Software">
  <meta property="og:description" content="Cut picking time 40%.">
  <meta property="og:image" content="https://example.com/og-cover.jpg">
  <meta name="twitter:card" content="summary_large_image">
  <meta name="viewport" content="width=device-width, initial-scale=1">
</head>
nginx — one hop, not a chain (www → apex, http → https)
server {
  listen 80;
  server_name www.example.com example.com;
  return 301 https://example.com$request_uri;   # single 301, never a chain
}

Good vs bad, side by side

✗ Bad — the staging flag that escaped
<head>
  <meta name="robots" content="noindex, nofollow">  <!-- oops: on production -->
  <title>Home</title>                               <!-- same on every page -->
</head>
✓ Good — deliberate, unique, machine-readable
<head>
  <title>Warehouse Automation Software | Acme Robotics</title>
  <meta name="description" content="Cut picking time 40% with Acme…">
  <link rel="canonical" href="https://example.com/warehouse-automation">
</head>

Continue with the other divisions

AI-ReadinessCan AI assistants ingest, trust, and cite your content when they answe…SpeedHow fast your pages load and respond — on the phone first, then the de…FoundationsThe infrastructure under the name: DNS, registry health, and whether y…SecurityThe connection, the content and the headers that protect both — and th…ReputationHow the wider web vouches for you — blocklists, breach databases, cert…

← Analyze your domain