wtf is
WTF is Crawling?
Google's robot reading your site.
In plain English
Googlebot visiting pages and following links to discover what exists.
Why you should care
If it can't crawl a page, the page doesn't exist to Google.
How to tell if it's happening
Search Console's Pages report tells you what Googlebot reached and what it skipped. If a page shows as 'Discovered — currently not indexed', Google knows the URL exists but hasn't bothered to read it, which usually means it found nothing linking to it worth following.
What blocks it
- A robots.txt rule you forgot about
- Content that only appears after JavaScript runs
- A page nothing links to — an orphan
- A server that times out when the crawler visits
Crawling is the first of three gates. Crawled, then indexed, then ranked. Fail the first one and nothing downstream matters.
Related
Still confused? That's fair. Call me and I'll explain it with your actual site open.
Keep going.
Everything on this site is free to read, including the prices and the parts that didn't work.
Or skip the reading and have me do it: (502) 963-7305 · send me the details.
