A head-scratcher to start the year with (luckily a friend's client,...

The Best Of

Go to the Best Of the SEO Community.

Noah

Jan 19, 2025, 11:21 AM

Forwarded thread from another channel:

Edward Bate

Jan 6, 2025, 10:04 PM

A head-scratcher to start the year with (luckily a friend's client, not mine ????)

• Site tanking off a cliff since early Dec, like -90-95% type clicks/impressions/traffic drop bad. Doesn't seem algorithm-related, and no major code/site changes made around that time

• Losing most indexed pages to 'soft 404' status - all different types including homepage, product page, blog page type stuff etc.

• Testing pages in GSC inspect, rich results test, SF crawl, view rendered source etc. don't flag any issues, nothing appears off

• GSC Crawl Stats all appears fine in terms of response times, response codes

The soft 404 part seems to indicate to me that some aspect of page content is not being rendered or displayed (e.g. Google is just seeing header, footer and blank in between) but the fact we can't replicate this in GSC inspect testing, and that it's occurring across nearly every page of the site (not localised to one template type or module etc.) doesn't support this? Likewise there's enough text and content in the raw HTML (Even if it is a JS-heavy build) so Google shouldn't be seeing fully blank content or anything.

Cloudflare is in place with some Googlebot spoof detecting / protection, but assuming that Cloudflare haven't stuffed that up, and that if they had it would be showing a bunch of server errors in crawl stats anyway? Possibly another layer in the tech stack somewhere is causing problems?

What would you be looking at next?

Emina Demiri-Watson

Jan 7, 2025, 12:28 AM

That's a head scratcher! I'm presuming the website does cover a topic/service that has a bunch of words/phrases that could confuse it ie, errors, not found… I think you can check that with SiteBulb. Also did you verify how its seen using another tool or just GSC?

Richard Barrett

Jan 7, 2025, 3:06 AM

Sounds to me like a cloudflare issue, they probably are allowing traffic through via the inspection tool but blocking genuine Googlebot at a guess and serving them a message.

You could potentially see this in the crawl stats section of GSC, under the settings side menu item

Could be intermittent rendering failure too - unfortunately the way to test this is basically to refresh the site a lot to see if you can get it to break, I would usually crawl a handful of pages I know were failing repeatedly and capture a screenshot/HTML of the page if your tool can.

If you do a site: search of your website do you get any weird content titles or descriptions pop up?

Shawn Huber

Jan 7, 2025, 7:30 AM

When you use Screaming Frog, which user agent are you using?

One client had this issue; they were allowing some Googlebot agents but not all, so the inspection was working as expected, but they were blocking other Googlebots that, when used in SF, were being given 403 status codes.

I also use this script as a bookmark that will render a page side-by-side with and without JS so you can compare

Shawn Huber

Jan 7, 2025, 7:31 AM

And if you don't have this, add it! It nicely overlays when the Google updates occur on your trend lines making it easier to investigate if an algo update might be behind this.