Google's Global Update: Impact on Rank Tracking Revealed
Go to the Best Of the SEO Community.
Derek Perkins
Jan 16, 2025, 12:21 PM
for anyone doing rank tracking, there was a major update Google rolled out globally last night to try and block crawling. This is a thread to dig into the changes:
Where you could use a simple cURL request before to get a SERP, now it returns a bunch of javascript with this message (html included)
> If you're having trouble accessing Google Search, please click here, or send <https://support.google.com/websearch|feedback>.
Derek Perkins
Jan 16, 2025, 12:32 PM
if you click on the link in the html, it does a redirect that adds an `sg_ss` query parameter
Derek Perkins
Jan 16, 2025, 12:33 PM
here's a curl request with that parameter included
Derek Perkins
Jan 16, 2025, 12:36 PM
that curl request succeeds with every header removed if the `sg_ss` param is included, and fails if it is removed
Derek Perkins
Jan 16, 2025, 12:38 PM
I tried decoding it using Google's standard base64+protobuf encoding that Google uses for most other things (<https://valentin.app/ved.html|description for veds> from @pletzer), but it's not valid base64
Derek Perkins
Jan 16, 2025, 12:39 PM
I'm still digging in, just thought I'd share this in case anyone else is working on it
Derek Perkins
Jan 16, 2025, 12:39 PM
this is the reason that many (most?) rank trackers are having issues today
Ryan Jones
Jan 16, 2025, 12:53 PM
this explains why none of the 3rd party APIs i get search results from are working today, and why my own tools are not working.
Derek Perkins
Jan 16, 2025, 1:02 PM
exactly
Mary Albright
Jan 16, 2025, 1:04 PM
dangggg
Valentin Pletzer
Jan 16, 2025, 1:06 PM
I’m guessing not many tool provider would be comfortable sharing if they have problems. If Google now starts to crack down on scrapers this would be huge. I guess @barry could ask around.
Derek Perkins
Jan 16, 2025, 1:08 PM
that's why I'm trying to be transparent, it's useful for everyone
Derek Perkins
Jan 16, 2025, 1:17 PM
crawling with Chrome should work, that just increases scraping costs 1-2 orders of magnitude
Barry Schwartz
Jan 16, 2025, 1:21 PM
Well one tool provider did <https://x.com/TomekRudzki/status/1879894440736895473|https://x.com/TomekRudzki/status/1879894440736895473>
Tomek Rudzki (@TomekRudzki) on X
We've observed a decrease (approximately 2 percentage points) in AIO detection rates through Ziptie's tracking system. It seems Google is blocking AIO checkers in a smart way, far beyond traditional captchas. We are looking for ways to improve the AIO detection rate.
Barry Schwartz
Jan 16, 2025, 1:24 PM
I’m asking other tool providers. Will keep you posted
Derek Perkins
Jan 16, 2025, 1:29 PM
<https://valueserp.statuspage.io/incidents/v23d1sv7yb75>
Google Web Search Service Disruption
VALUE SERP's Status Page - Google Web Search Service Disruption.
Derek Perkins
Jan 16, 2025, 1:34 PM
I was hoping to reverse engineer that query param to see if we could skip that redirect page
Ryan Jones
Jan 16, 2025, 1:35 PM
and sadly valueserp is used in my tool SERPrecon - so we're down ????
Ryan Jones
Jan 16, 2025, 1:57 PM
@derek @barry found any other tools impacted? all the major ones tell me "we aren't aware of any impact"
Derek Perkins
Jan 16, 2025, 2:06 PM
yes, but not at liberty to share names. I'm skeptical of ones saying they don't have issues
Derek Perkins
Jan 16, 2025, 2:12 PM
I just did a manual check with Accuranker and got new data, and I _think_ DataForSEO is working
Ryan Jones
Jan 16, 2025, 2:12 PM
dataforseo seems to be working. at least in their playground
Ryan Jones
Jan 16, 2025, 2:13 PM
SpaceSERP claims they are up
Ryan Jones
Jan 16, 2025, 2:15 PM
but their playground does not work
Derek Perkins
Jan 16, 2025, 2:15 PM
I just ran searches in both their playgrounds too, same results
Ryan Jones
Jan 16, 2025, 2:17 PM
i wish scrapingrobot had a playground i can test. there's no documentation for their SERP API but i know they have one
Derek Perkins
Jan 16, 2025, 2:18 PM
DataForSEO already had some more expensive options to get pixel data and whatnot, that likely used Chrome, so I wonder if they're routing all requests through that infrastructure for now
Ryan Jones
Jan 16, 2025, 2:18 PM
can confirm BrightEdge instant is giving me results. so is Moz rank tracker. no idea if they're cached though.
Ryan Jones
Jan 16, 2025, 2:19 PM
not aware that semrush/ahrefs have real time SERP tools.
Barry Schwartz
Jan 16, 2025, 2:59 PM
I emailed several asking for statements
Ray Grieselhuber
Jan 16, 2025, 5:29 PM
We're not having any issues, we do mostly JS though when scraping
Ray Grieselhuber
Jan 16, 2025, 5:31 PM
<https://theseocommunity.slack.com/archives/C05J031AJ81/p1737058017923299?thread_ts=1737055263.703179&cid=C05J031AJ81>
Transparency is definitely the best approach here, agreed with @derek. We're just not seeing much impact yet on our end.
Derek Perkins
Jan 16, 2025, 5:38 PM
are you scraping with js enabled?
Ray Grieselhuber
Jan 16, 2025, 5:38 PM
generally, yes
Derek Perkins
Jan 16, 2025, 5:42 PM
I think that's the major differentiator for this, Google started requiring js if certain conditions aren't met (cookies and/or query params)
Ray Grieselhuber
Jan 16, 2025, 5:42 PM
Yup that's what it looks like
Patrick Stox
Jan 16, 2025, 7:45 PM
Commenting to follow, but I'm not seeing any issues in the data, or customer complaints from Ahrefs users. I just checked and we have recent SERPs and can pull live SERPs.
Patrick Stox
Jan 16, 2025, 7:50 PM
If we had any issue, we'd usually have hundreds of complaints by now.
Derek Perkins
Jan 16, 2025, 10:03 PM
<https://www.linkedin.com/posts/nataliawitczyk_fresh-in-google-starts-intensifying-its-activity-7285764251690946560-ZdVp?utm_source=share&utm_medium=member_desktop|https://www.linkedin.com/posts/nataliawitczyk_fresh-in-google-starts-intensifying-its[…]251690946560-ZdVp>
Natalia Witczyk on LinkedIn: Fresh in: Google starts intensifying its anti-scraping measures… | 32 comments
Fresh in: Google starts intensifying its anti-scraping measures, introducing stricter protections such as IP blocking and CAPTCHAs. Popular SEO tools like… | 32 comments on LinkedIn
Barry Schwartz
Jan 16, 2025, 10:51 PM
@patrickstox can I quote you?
Barry Schwartz
Jan 16, 2025, 11:24 PM
Actually I see you commented publicly on LinkedIn. That works for me
Barry Schwartz
Jan 16, 2025, 11:33 PM
Posted about this at <https://www.seroundtable.com/google-blocking-seo-rank-checking-tools-volatility-continues-38759.html|https://www.seroundtable.com/google-blocking-seo-rank-checking-tools-volatility-continues-38759.html>
Barry Schwartz
Jan 16, 2025, 11:34 PM
Thank you
Derek Perkins
Jan 16, 2025, 11:35 PM
I wonder if the lower volatility in those charts is because those tools haven't gotten as many SERPs recently...
Seppo Puusa
Jan 16, 2025, 11:36 PM
I also think the change in volatility metrics is partly due to data issues
Barry Schwartz
Jan 16, 2025, 11:37 PM
Yes. That is what I am guessing. But I’m wrong about 91% of the time.
Seppo Puusa
Jan 16, 2025, 11:37 PM
@barry I should steal that line and use it with every client recommendation from now on
Barry Schwartz
Jan 17, 2025, 4:15 AM
<https://twitter.com/rustybrick/status/1880212123432939890|https://twitter.com/rustybrick/status/1880212123432939890>
Barry Schwartz (@rustybrick) on X
On Google blocking rank tracking tools - @RangerShay from @Similarweb sent me the following statement which has been added to the story at <https://t.co/7Z0MIy9Ckh>
Seppo Puusa
Jan 17, 2025, 4:16 AM
Data for SEO SERP API timeouts at the moment.
Derek Perkins
Jan 17, 2025, 6:56 AM
this is a good perspective I think
<https://x.com/dejanseo/status/1880210403105616004>
DEJAN (@dejanseo) on X
@rustybrick PS: This has nothing to do with rank trackers specifically and all to do with LLM training data scrapers.
Ray Grieselhuber
Jan 17, 2025, 7:09 AM
We ended up getting blocked last night around 9pm ET, but we're starting to look good this morning.
@derek I completely agree with that post. I started noticing Google results (instead of Bing) in ChatGPT results and I think Google was not amused.
Derek Perkins
Jan 17, 2025, 9:44 PM
This outage made it to TechCrunch
<https://techcrunch.com/2025/01/17/google-begins-requiring-javascript-for-google-search/|https://techcrunch.com/2025/01/17/google-begins-requiring-javascript-for-google-search/>
Google begins requiring JavaScript for Google Search | TechCrunch
Google says it has begun requiring users to turn on JavaScript in order to use Google Search, possibly in an effort to block certain SEO tools.
Written by
Kyle Wiggers
Est. reading time
2 minutes
Derek Perkins
Jan 19, 2025, 9:51 PM
@ray are you seeing iPhone requests rejected more often than Android? I can't even get a search to load on my machine if I use Chrome emulation
Ray Grieselhuber
Jan 20, 2025, 7:18 AM
@derek just checked on this with our team and tested on our live serp tool - looks like we're fetching it with no issues
Derek Perkins
Jan 20, 2025, 10:58 AM
thanks for the confirmation
Ray Grieselhuber
Jan 20, 2025, 11:14 AM
Sure thing
Channels
Join The Community
The SEO Community is a free organization, with private Slack group and events, for those interested in search and AI optimization to learn, discuss, and meet others in the industry.
Powered by MODX
Resources
The Community Story SEO Articles SEO Jobs SEO Conferences Top SEO Discussions Newsletter Free Tools Learn Claude Code
The SEO Community © 2026
Shop Merch Code of Conduct Partnerships Sitemap Update cookies preferences Terms of Service Privacy Policy