Find out if Googlebot can crawl and index a page. We fetch the URL as Googlebot and as a normal browser, check robots.txt, noindex tags, canonical, redirects and status codes, and tell you exactly what blocks the page.
| Googlebot | Browser |
|---|
Crawling means Googlebot downloads your page. Indexing means Google
stores it so it can appear in search results. A page must be crawlable before it can be indexed, and
even a crawlable page stays out of the index if it says noindex or points its canonical
tag at another URL.
Disallow rule in robots.txt that matches the URL.noindex robots meta tag or X-Robots-Tag header.