Show Transcript

Noah Learner

Good morning, good evening, and good night. Welcome to another edition of the Campfire Chat. Today, we're incredibly excited to welcome Dan Petravich, who is literally burning the midnight oil. It is 12:01 a.m. for Dan. He's in Australia. And today is incredibly exciting for me because I've been hearing about Dan for years from a friend of mine named Dave Sodtomano and I just felt like we had to connect.

Before we jump in, I want to thank our community sponsors who are just amazing. Hrefs, Air Ops, Citation Labs, Jet Octopus, which is really exciting because they just joined us. DriveFly Digital, Demand Sphere, Bento, ModX, and SEO jobs from Nick Leroy. And he had a huge announcement because he just got accepted as the official jobs partner for Search Engine Land, which is really cool.

So, for those of you who don't know about Dan Petravich, let me give you a little bit of a background. And my background's going to stink and I'm loving I'm really looking forward to his, but for a bunch of years Dave Sodtomano who's been railing against folks who are just doing best practice work and not really doing anything in the field of innovation about learning what Google is or where it's going or how it works and he's been talking with me for years about Dan Petrovic and I've been a weenie and haven't reached out to you yet Dan and so I was really excited when I finally did that you said yes to join us here in a campfire chat.

I think we're at a pivotal pivotal time in the in the industry. When I talk with SEOs, the conversations now are really different than a year ago or two or three or four years ago. You know, that storyline arc is one of going from huge amounts of hope about the future, loving amazing salaries that they were getting, opportunity and growth to one of just like quiet huddled fear and night sweats, right? And a lot of it is this transition into what is the future of search? What is our search surface going to look like? How are we all going to be impacted in our day-to-day work? What are the tools? What are the strategies? What are the actual hard skills that we're going to need?

And so that's what brings us together because I feel like you've been doing more work than pretty much anybody to answer these specific questions. So I'm hoping you can kind of shine the light for us and make us all smarter about what the future looks like. Before we get there, it would be great to dive into your history because you've been in SEO forever. Can you bring us up to speed? Maybe talk about how you got started and maybe up to about 2012.

Dan Petrovic

Yeah, look, start a little bit earlier than that. Imagine 2001, underground rave warehouse. I have blue hair. You can still see some knobs up, you know, I was tweaking knobs in this warehouse. Police comes, breaks up the party. This is where I was at. So, I had this website for electronic music and I started to optimize it this and that. Next thing you know, I'm an SEO. But before that happened, how I actually got into it, people offered, like some German guys offered to buy some links from me. And I'm like, buy a link? What do you mean? And then I realized that they want to rent links from me for like $500 a month. I'm like, what is going on? Oh, you've got page rank seven, page rank eight. Fast forward, I've got like a thousand domain link network that nobody's ever heard of. Super private, really well tuned. Paid off a mortgage, started off an agency and that's when the Jean kicked off. The Jean as an agency grew very big and then I dismantled it because I don't like big agencies. Yeah that brings us that brings us like a super very quick you know fast forward that brings us to 2012 2013 my daughter was born and I'm like okay because I'm a sci-fi guy and I love futurism and I love predictions and the guy who inspired me was actually Rand Fishkin and he made some bold predictions and then checks in on himself to see how he did. It was usually a year-on-year prediction. I said, all right, let's try to predict the future 10 years from now, like Isaac Asimov or Arthur C. Clarke type stuff. So, I wrote an article called Conversations with Google. I said Google will be a chatbot. We'll be chatting to a search engine. It'll have agentic features. And I proposed a whole development timeline of Google, which is on track now. Got a little bit lucky with timing. We can thank OpenAI for that. And here we are. My love of artificial intelligence and machine learning was always there, but I couldn't code. I'm really bad at math and people think I'm a technical guy. I'm not. I'm just really creative and persistent. Um, so what what happened is like you know I got wings like suddenly I've got AI doing coding for me basically and I always have dev people work with me but it's a bit different like it's a bit more direct it's kind of like you know that the power is straight to the vein direct access to to um coding so I don't have to go through intermediaries and so I'm like this I was I've always been hungry for this and like when Chad GPT like the early days of GPT were painful like short contexts back and forth really bad but now like it's competent like a test I've test driven GRO four today um the thing is smooth it's not the best conversationalist doesn't collaborate as much as um perhaps Claude Claude has fantastic personality design um but yeah um you know it's just the sheer bottled up desire to develop and innovate and do that I couldn't do before that's opened up now. So gen is out of the bottle, and I'm just like it's this un unstoppable desire for innovation that I couldn't do things that I can do no,w and it's it's compul it's compulsive I have no choice over it. My philosophy has been as soon as I discover something, I just share it straight away. And I think that's what people have been noticing. SEO is not boring for me anymore. All that best practice stuff, SEO hygiene, I was retired from that.

Noah Learner

I wanted to get to that because you quit for a period, right? Yeah. I was just like there's nothing for me to do anymore. Agency is running itself. What do I do? So I was like doing science and music day myself dad for like um for like 10 years you know um spending time with my daughter playing and things like that but then work got really interesting um so I'm like okay let's do some fun things and that's what's really happening now and I'm trying to engage the community uh perhaps one thing I'd like to send out there to whoever's watching is like sometimes people see my stuff and if they like it it seems seems a little bit intimidating, but it's not meant to be. I'm not I'm not trying to scare anyone or or present SEO as like too hard now. We overthinking, over complicating things. We're doing research, a lot of research, and we're discovering things. And my best mode of collaboration is like I put something out and people jump on it, criticize it, improve it. Oh, did you see that? Did you see that? Oh, that's already done. Basically jam with me on these ideas and then that knowledge that I've just gained and shared amplifies with the community. So Dave has been one of the people who jump on and go either oh actually no you're wrong because this and this and that and I go okay cool yep no harm in somebody like prove me wrong like I love that sort of stuff, or like did you see something else really cool that you know might do a better job or simply adding to ideas. So, of course, Dave's not the only one. There's like a whole group of people who are jumping on board and trying to make sense of everything on the same level that I am. But there has been a bunch of people who've reached out to me and saying, "What's going on? What is all this, you know, like almost scientific stuff deep? Is this really necessary for us to do our jobs?" It's not. You can still do your traditional SEO, but it's like there's this very lucrative area to look into, like LLM interpretability, mechanistic interpretability, which is one thing that I'm really tired of is like us worshiping these Google algos like gods, and praying to them and fingers crossed kind of mentality, you know, passive observers, non-player non-playing characters in the game. We now have the ability to actually participate, to engage, to understand the models on the very very low level. The data is there. Yes, it's a little bit chewy. The subject matter is a bit chewy and there's a learning curve, but the reward is there and it's really really exciting. So that's my take on where we were and where I'm at now. And so stuff that I'm putting out recently may seem a little bit random and hectic and overly technical, but it's not. I have no background in any of those things. So it's like my, I built up my knowledge of machine learning within three years. That's it.

Noah Learner

Okay. So let's take us through your agency's arc a little bit. When I looked for your content and I looked at a lot of the decks that you've presented in webinars, it was around links and link building. That was the old, that was like a lot of your old stuff. Was links and PR, was that core to agency services for you historically?

Dan Petrovic

Yeah, made us a lot of money. We had we had a we still have a link building living thing. We do we work with journalists as well. Outreach is not gone. Outreach is still a solid element we rely on. But we've over time we started off primarily before the agency like I said we had like a link network that was probably the most profitable business model I've ever had. And then I really wanted like I didn't believe in that to be the long-term solution. I was actually wrong. I should have just continued. Like I dismantled that a bit a little bit too early. But anyway, so I wanted a proper agency. So we transitioned from the link network to link building, from link building to tech SEO, from tech SEO to PPC and everything else that we wanted, and then we ended up being a strategy first agency. And then we went from a cycle of being quite big, having like 60 people in total down to like a cap, the number of people we can have full-time employed to 10 because like after 10

Noah Learner

you have to have management reports, right?

Dan Petrovic

Yeah, exactly. It's it's it's and um I'm like not sad to say, but I'm actually really pleased we no longer employ juniors. I've done that for 20 years. I've trained people and those people have gone on to other agencies and so we're done now.

Noah Learner

Um nor you have basically the powerhouse team in Australia, right? Yeah. I saw a bunch of folks on the team. I was like, whoa, I didn't know they were there.

Dan Petrovic

Yeah. And so like when you look at for example like Nick Ranger and her skill set in terms of technical SEO, the types of projects that we work on, it's the type of stuff like people people run out of ideas and they get stuck on really difficult technical projects and like they've tried everything and they've come to us and we we try to fix those types of things. So really all senior team and that's that's where the configuration is. The work is really hard and really difficult and we don't have that the luxury of a production line pipeline because a lot of projects that come our way are always bespoke, always custom, always like some specific thing and so we end up putting a lot of hard work into it and we have to keep reinventing things. So like I think every single project we have.

Noah Learner

It's so hard to be profitable. Like you must charge a ton. You must charge a ton.

Dan Petrovic

Yeah, we do. We're not we're not cheap. But what happens is like every time we complete a project for a client, we end up with one, two or three new tools or processes or workflows. So the amount of development we've had in the last I'm going to say three years has been phenomenal. Okay. Just just the tech stack we've built as a result has been the value that we didn't see in the financial sense.

Noah Learner

Yeah. So, I know it might feel to you or the audience like I was going way off track by asking about the link building piece, but I promise it's connected. And here's how. I know a lot of link builders who are afraid about the future and I know agencies are also acquiring link building agencies and the agency that comes to mind is previsible. They just they just purchased I think internet marketing agent uh in internet marketing ninjas and huge acquisition for them.

Dan Petrovic

Yeah.

Noah Learner

And I think that the narrative there and their strategic move and why I think you're well positioned or at least I want to get your sense of this is link building equals credibility, credibility equals source material for LLMs to crawl to then build confidence about brand and that that's what's necessary to appear in LLMs. Can you connect the dots for us smarter than I did?

Dan Petrovic

No, you you pretty much. Like I said earlier, link building is not gone, but it's also not in the same shape or form as it was. I think one area where link building teams who might be panicking, you said a lot of SEOs are panicking. There's this nervous energy. Everyone's putting on a brave face and trying to look confident but there's this underlying sense of nervousness and fear like you mentioned. But I think we're at a historical moment in our industry and I'm not scared I'm really excited about everything that's happening because and I'm going to probably take a page from Mike King's book here and he says and we align basically on like SEOs have not being valued enough and this is our chance to shine. Now we do a lot more than people give us credit for and now it's quite obvious once we take ownership of AI visibility people going to go okay SEOs these guys are now worth a lot more to us because of what's coming. And SEO's, like AI is really hot right now, everyone's, and that's a bit of a risk as well but anyway back back to outreach teams I'm going to call them rather than link building teams. I think naturally these teams are really well positioned to seed content on the internet and they don't even necessarily need to get links anymore for as long as they get the right type of content seeded about the clients that will influence LLMs. However, there's a huge disconnect in what people think that they need to do and what they actually do with this type of outreach practices today. The truth is that nobody really knows and that's one of my primary areas of interest right now. Focus. Which buttons do we press to get LLM to speak favorably about our clients brands? Noah Learner Yeah. Yeah. That's that's an overimp simplification. I could talk we could talk for three hours about this topic. So mechanistic interpretability is one of those areas where we try to understand what is actually happening inside the models.

Noah Learner

I want to go deep there because I feel like that's the question that we're all trying to answer and people just don't know how to experiment around it. So, I want to dive into how you actually think. One of the things that I love most about these conversations is learning how the smartest people in our field think and how they approach the work and how they mentally think through things and problem solve. So, take us inside the mind of Dan Petrovic. Help us understand how you think, how you approach problems, and how you approach this experimentation.

Dan Petrovic

Yeah. So there are two types of models. There are open source models and there are closed source models, which are the commercial models. So we've got open source stuff like Gemma, which is fantastic. I'm really grateful to Google for that. Gemma, Mistral, and we’ve got great models coming out of China. China’s been fantastic. We've got a really strong open source community in machine learning and large language models that we can probe very directly. On the other side, we've got commercial models of commercial size. Even very large models that are technically open source, like Llama, the gigantic 600 billion parameter version it’s like crazy

Noah Learner

Yeah, insane.

Dan Petrovic

Yeah insane size um so that's while while open source it's really not like you can't do anything with it unless you have a cluster um so but let's let's call it are still impractical to work with unless you have a cluster. So let’s call the commercial group Grok, Claude, Gemini, and GPT. If we look at those, you can’t get into the model’s head. You can’t look at the layers. You can’t examine its activations. You can’t probe into the model in a scientific way. So what we can do, and what most people do, is survey the models and analyze the results. That’s what these AI trackers and rank trackers do. I actually don’t understand how this became such a big business model, but people are willing to pay money to do that. I figured like that's just a script that runs. You just prompt your models once per day or once per week and then you collect the data and you do natural language processing and but like not everyone is hungry to develop all that internally. So and so that there's market for for that type of stuff. However, uh probing models as a simple survey and getting the results back and kind of like trying to do analysis not enough. That's that's not uh good enough. You need to go a little bit deeper. Um so our team goes to the level of log props. So for every API completion we look at every single token. We um understand what are the four tokens for that token. So we have five tokens um in in the sentence completion that could have been predicted and we try to follow that tree. So that's a tree of probabilities of every single thing the model could say about a brand and then we analyze um all those probabilities using dynamic thresholding.

Noah Learner

I want to get into this. I was reading your post and it seems like you're struggling with this now like you engineered some crazy solution. you know, fingers crossed. I hope it works. And then the next series of posts that I see from you is basically like, help. Like, who's out here who can help me think through this?

Dan Petrovic

Yeah. Yeah. Um, look, the the the stuff that I'm describing now, we call it tree walker. So, it's effectively a dynamic thresholding algorithm that gets API output and dissects it into log props of each individual token in this in the completion sequence. So if I start with Nike as the brand and then I let the model complete the sentence Nike is a company that blah blah blah blah. So we look at all those things and and what we do is we collect the log props for every token in the completion. We walk the tree. We walk it. So we don't go five uh 1 5 25 125 like because 20 steps into it average sentence you'll be like into billions of potential sentences. So what we do is we do we do a reasonable tree walker um and we follow we follow the potential completions and path pathways we follow the pathways at junction points that are above a certain threshold point and that's the stuff that I've been thresholding how do you set the right threshold point when 159 you're doing the reasonable tree walker in the in the um API response completion tree because the tree of probabilities is vast but only a small subset of the entire entire tree is actually meaningful sentences. Believe it or not, the rest is just garbage. It doesn't even make coherent sentence, repetitive things and all sorts of things. So the next step in this um in this closed source model um that we do is we look for high entropy and low entropy points. What that means is um low entropy points is like when model was always going to say what it said. High entropy points is like um model could have said one thing or the other about your brand. uh you could have said one product or the other, one service or the other or three different things with a probability of 30 30%. Uh and so we look at those we we look at those u weak spots that connects with the link building team's work with the outreach team's work. What are the little things that need identification? Little things that need attention because what you say at that point will influence the completion of the entire sentence after that. And what is it that model is not confident with? So this is now done without grounding because you're not getting rag pipeline into the model. Model is giving you the answers out of its head.

Noah Learner

Right? So this is a portion of people that don't know what grounding is. So can you just define that so that everybody gets smarter?

Dan Petrovic

So grounding is effectively providing search results to a model before the model answers your question. Simple as that. So it gets to see search. Um and grounding can be done in different ways. You could provide URL context. You could provide your internal data documents this and that. In terms of AI mode and AI overviews and and Google the new version of Google um and even GPT when GPT provides answers it grounds its responses um each paragraph each passage is grounded with an external resource. Um Google does it a it's a little bit of a specialized topic Google does it in in reverse they do the generation and they plaster the uh u the citation on top to make it look legit. Um but what they should do is really generate from search results um rather than patching it. U but that's a separate topic. I'm critical of Google doing that. That's not the right way to go about it. They know that. Um I know that they know that it's I think they might fix that in the future. Uh where am I going with this? So grounding. Yeah, grounding is providing providing models with real world data so they don't have to rely with their their internal memory of the world. But I'm interested in not just so your traditional SEO influences the grounding data. But I'm interested in changing models opinion about a brand internally, viscerally, natively within the model itself. So if you switch off the internet, the model knows the brand. So this is where these little brand surveys come in handy and tools like our we we have a free tool that we u published for everyone to use AI rank. Um and so that tool does two things. It asks the model what does your brand do and then it lists the top thing top 10 things and then we ask for the things that you listed what brands can you think of and we collect the results and we measure the position and we measure the frequency. We we normalize those two values and balanced them out and come up with a weighted score that is like the concept the entity or the brand visibility um in that context. So that's just birectional model probing. So we still we're still in the realm of uh opaque blackbox models. This is as far as we can take it. U natural language processing probing um probabilistic analysis tree walker really clever stuff when you think about it. The way we the way we cut the tree to only the branches that are meaningful and analyze that data and then give that give that intel to the link building teams and outreach teams to work on it's way more efficient than just flailing about at random just like I just you know do do some stuff link builders do some stuff outreach people.

Noah Learner

Um okay I love it okay so couple questions the first thing I'm hearing or wondering because I just am have not done all the stuff that you have. So what I'm hearing is that we can understand what a model knows. What a model knows in its native sense as you were describing it is based on its training data and and it doesn't but that doesn't change until the next version comes out. Right.

Dan Petrovic

Correct. So fine tunes happen perhaps once a month. Half versions happen six month like like maybe twice a year and new model releases like you know from GPT3 to four that might happen over a year or two.

Noah Learner

Okay. So let's say I'm an agency and I'm doing strategy or an organization I'm doing strategy. What I'm hearing is if I want to get in the native version I can know what's possible. I mean, I know what the I can find out what they know and then I have x amount of time to create a new version of what they know by changing my presence online through outreach to become more credible. Um, but it's almost like a Google algorithm update where you make changes and you're hoping that the next algorithm update takes that into account and you see like a sitewide boost kind of thing.

Dan Petrovic

Yeah.

Noah Learner

Is it the same kind of strategy where it's like, okay, they know X about me. I need to change that knowledge through how I appear both inside my website and outside my website. And in so doing, I can hope to down the road appear better natively. If we're getting rid of grounding as as you know, if we're getting rid of grounding, that's how we can think about appearing just in straight LLMs.

Dan Petrovic

Yeah. You you can't really almost you can't really disassociate grounding with the model responses because in most realistic practical situations it's always there.

Noah Learner

Yeah. And okay

Dan Petrovic

Uh more than likely the model will be grounded especially when we're talking like AI mode and and emerging search tech like GPT and so on. They'll they'll ground so we're not really worried about that. So between the model updates, the grounding will carry us through as far as visibility goes. But um one thing that I did notice, like I mentioned, I have a I have a tool we have like maybe two or three thousand users. So we collected a lot of data. Um yeah. And so what one thing that I've seen is when you me when you monitor brand um visibility across like time series type data what you notice is that for example brands like you know Nike and Adidas they're just like stuck at the top and they're always at the top every week we probe the models they're always at the top whereas other lesserknown brands they oscillate sometimes they're in this position sometimes in that position, but like the big authority ones are stuck to the top and they hardly ever move. So, we actually found an interesting metric that the the amount of oscillation that the that you that you notice in the rank tracking of a brand in LLMs is directly associated with the brand authority. So, it's like a really really cool elegant little metric. More it moves lesser authority lower oscillation higher authority super cool stuff. So that's one thing. Um and um obviously like when you notice um that your brand is like fluctuating wildly that actually says the model's not really sure about you um and they're trying to mix up. But when the model is really sure that there are certain certain authorities that there'll always be presenting that. That's what you're trying to change in the model's head like I say. So what what that means is that when when grounding is applied to the model and it's evaluating the results obviously there's position bias especially because machines are even like more favoring the top results than than humans. So um there's there's like a high likelihood that the model might pick out something from that grounding from that rag pipeline and pick that out over something else if it recognizes it um from its pre-training. So imagine so this is exactly like the type of stuff we did with u click rate optimization. We knew for 10 years, for longer, that if somebody recognizes a brand in search results, they'll click on it more so than on other brands. And so brand recognition becomes a significant factor in clickthrough rate optimization. Um to the point that it can sometimes even override the position bias. So you could have one result in position one, second result on position two, second result will get more clicks if it's a well-known brand. Um, of course that can be mixed up because like well-known brands tend to rank high as well. So you could you could have them. But theoretically, if you had a situation uh where you had um a better known brand rank below a lesser-known brand uh you would probably find high clickthrough rate than you would expect from from the click to rate averages on that position. Um so where are we going with this? We're now influencing both machines and the humans. And so machines are doing the the machines are doing the fan out queries that are happening in the background collecting all those results bringing them to the model and model is then decided what to pick out of that. So if the model recognizes your brain if the brain is really ingrained with the concepts relevant to the user's query or a prompt then that means that the model's more likely to pick you out of the lot. Imagine you're a model. You just you've just given hundred different results. Which one do you pick out as the most relevant? You'll pick the one that's yes relevant um seems relevant to the query or the prompt but also the one that you're confident in that clicks with your training data and that's that's one thing that we're trying to influence. Remember we're still just talking about blackbox stuff the stuff that we believe we cannot really open the hood and understand but so on the other side we've got open source models like Gemma and Gemma is super interesting and that's like that's the stuff that I've been struggling with recently with Gemma you can open the bonnet and look what's in there. And so like there are libraries like lit um and a variety of other like um model interpretability tools, mechanistic interpretability tools that allow you to see what happens at every layer of the model. That every activation, every connection in the model, the whole network, you can see what lights up during the completion of the of the whatever you I said give me recommend some brands that do such and such and then you can see the model as it lights up in different layers. I'm trying to oversimplify things so people would understand. So basically with the with these um open source models you can actually do that. Why Gemma? Because Gemma is effectively mini baby Gemini trained on the same tech. Um Google has a ce of the web internally that they use both for search and for LLM training and Gemma has been trained on that. So we can see like it's like a mini Gemini if you will especially if you look at Gemma Gemma 3 27 billion instruct model for example it's almost a little Gemini really. Uh I would say that that that model is more powerful than the Gemini we see in the inbuilt in Chrome.

Noah Learner

Can you I mean so getting ready for this talk last night I was supposed to be researching you and I ended up jumping into a Google's AI studio. Maybe Amy can you share that link um with folks? Um, and I was trying to understand query fan out. And I'm not saying query fanout is the thing that we need to learn about, but it seemed like the most interesting thing that I was hearing about at Google IO. And so I wanted to understand how it worked. And it felt like Mike King was on to something really early and we talked about it at IO. And then he he basically took us out to dinner, dipped, went back to the hotel room and built Qoria. I mean like that night. I mean like so um so I jumped into Gemini or AI studio last night and I wanted to understand how Gemini works. And the reason why I did that was that both Google and Mike and other folks are talking about how basically you want to understand AI mode you got to understand Gemini because it's Gemini 2.5 Pro is the guts of how it all works.

So, um, I went into AI Studio and I'm hoping you can kind of walk through how you use it or how you're using it to experiment because I basically went in and just said like, hey, I have this entity plumber. Tell me everything you know about that entity and return results in a structured JSON object. And it was like bam. And it gave me all kinds of stuff in that response that I didn't understand. And I started to ask questions about every single property and every single description that was inside the property. And I kept having it build and build and build a prompt that I could use to do intent analysis on any kind of entity that I wanted to rank for. And then I got it to use grounding. And then I got it to use all this stuff. And I got it to return um search results that it was using which was a pain in the ass to get it to actually share the URLs that it was actually researching as it was doing like grounding. And I learned through that process that when it returns URLs it's kind of difficult for it to do it and it has to think through it because it by accident returns results as like vertex AI search results that redirect to the actual URL. So you're like, so you start to understand the problem domain the more and more you experiment with it.

And I'm hoping you can kind of share a little bit of your journey with it and maybe help us learn how to experiment a little better with it.

Dan Petrovic

Yep. Do you have two hours?

Noah Learner

Um 10 minutes, 20 minutes, 30 minutes.

Dan Petrovic

Yeah. Look, uh I I the reason I'm saying this is because I like this this this this morning I had a a meeting with one of our US clients and um that was it that was the project. So we basically um did heavy heavy model probing. Um we spent we we got quite familiar with Gemini. Um and so I can take it through a summarization of the approach I guess.

Noah Learner

Yeah, that's cool.

Dan Petrovic

Yeah. Um, so you're right. You're definitely you understand it now. You understand like the vertex URLs that need to be followed. So we have we have like a little module that follows the reader. It's a three or two redirect. We we have a small little Python um function that is in charge of following the silly vertex URL and then we get to the final URL and then bring we bring that. But guess what we do with that? We don't stop at that point. Once we found the grounding like I think people if you're listening tune into this this is a golden nugget.

So um you follow the citation URL the grounding URL for the for the completion for the um response from the model you follow the grounding URLs you find the target then for that target URL you use URL grounding not search grounding URL grounding.

So that API gives you the clean, pure, clean version of that page's content the way Google has it. You don't need to scrape. You don't need to guess. It's exactly what Google has. This is like superpowers. I'm not sure.

Noah Learner

What is that tool, dude? I'm listening. Like what's the tool in AI Studio?

Dan Petrovic

AI Studio. Um, so this is like a little flick um little toggle switch you can switch on. You can switch on the the search grounding and there's another little switch there that you can enable URL grounding.

Noah Learner

Yeah.

Dan Petrovic

So like if you switch off search and you enable URL grounding, you can paste a URL and say get me the get me the content from this page and it'll give you the content from that page verbatim. If you say give me give it to me as JSON, it'll give it to you as JSON. So it's like scraping except not you just pay for the API call.

So what I'm doing what I'm doing is I'm just jumping in the middle of the process for some reason but I think it's a really interesting people love little actionable things like that. So like basically you get the citation you follow the redirect you find the grounding URL and then you get it's your you get this content but let's let's wind back. So competitive analysis you start with your primary brand that you're working on whether it be a client or you're working inhouse and then so what you do is you use the euro grounding tool and you get the content from the homepage of that website and then you do that same exercise for your nine other competitors or 10 or whatever. So you have a group of competitors that you've scraped or extracted clean Google C content for each one. And so this is what Gemini Gemini is really phenomenal at. So you can because it has a 1 million token context. So you get you get all those u pages um you structure them so it knows what's what's what which brand and then you can say give me a list of aspects that these brands are pushing on their homepages things that are really important to them. We are cheap. We are the best. We we focus on this service. We focus on that service. We focus on that product and so on and so on. So you can go back and forth jam with jam with Gemini like until you refine that list.

But once you have that list, what you do then is you then structure a model uh a model probing um thing that analyzes analyzes each one and builds you basically a feature important m importance matrix. So you can look at each brand's prioritization and things that they care about. So you marry that um in the next stage with the stuff that you've done through prompting. So for each brand, what does nike.com do? What does adidas.com do? So you collect all those uh completions and you get raw responses.

Do 10, 100 for each brand. So you basically do natural language processing on the response of the model on each one. Things that you can do with that, you can do named entity recognition. Great model that you can use for that is Glina. Gler gives you arbitrary labeling. So you don't have to train a model for every industry you work on. You just tell it what labels you'd like and you give each label a description and Glina just snaps them up and recognizes named entities within the responses of the models. So that's one thing you can do quite easily. And the other so that's part of our pipeline actually. I'm just literally describing our pipeline.

Um so another thing that you could do is you can do pretty straightforward sentiment analysis. Um how models think about u different brands. Um you can extract all the citations as domains and the URLs and guess what those URLs are when like once you once you have let's say you do 100 completions for each brand competitor and you compile this vast list of grounding URLs that list is the prospecting list for your outreach team. Let that sink in. So basically, you built yourself a link building or or digital PR list with with no human work involved. You basically just been probing models, collecting citation URLs, following them, and getting um and getting like your list is done for you. Um it's it's it's an amazing um opportunity. And this is all there's you're not breaking you're not doing any black hat. You're not breaking any guidelines or rules. Just following following the rules like everyone else, paying for the API calls and getting huge amount of insight out of that. Um, so yeah, I probably I'll probably park that idea for now because there's a lot more to talk about, but um, yeah.

Noah Learner

Okay, we got some questions coming through the audience that I've been neglecting. So, let's get to it.

Now that the buzz has died down, how do you feel about Google's leak now? Do you think user behavior metrics will drive AI rankings as well, or will mentions and coverage, etc.?

Dan Petrovic

Yeah, I forgot about the leak. Um, yeah, that's like one of those things I try to like I've been honest about. I try to stop the leak. Um, I I try to shut it down because you don't tell Google that you have it. I was gonna I was going to circulate it discreetly um throughout the SEO circles, like trusted circles, but um um it just got out like genie was out of the bottle and you know, I was like I was like, "All right, let's let's do it. Let's jump on it." Um so I I spent a phen I spent three months analyzing the data before it was leaked publicly. Um, and I have like I have a search engine for it. I have some original data. I spent I spent a lot of time thinking about it and looking at it. But ultimately what I found is what I already wrote about. There was there's an article on Moz that's called I published long time ago that's called user behavior as a ranking signal. At the time when I published that some people were like oh this is really cool. Maybe it is, maybe it's not. I'm like, "No, no, it is. It is." Chrome, Android, type in disk. Type in histograms in your Chrome in your Chrome URL. You'll see all the data that's being sent. Every click you make, every copy, every swipe of the text, every button press, every single thing is zipped up and sent to Google. Unless you untick that button saying, "I don't want to send it to Google."

Noah Learner

Yeah. So, I knew that and we looked at that with Cindy Creme. We looked at histograms with Cindy Creme.

Dan Petrovic

Yeah. Um, so basically, um, I proposed that idea and Renfishkin's been talking about it and he's been ridiculed and people were laughing about, you know, and and and then suddenly we find it in the source. Um, user behavior signals and yeah, they're there and they're in Google's DNA. You can't just rip it out. You can't just patch it up quickly because that's that's that's how it work. Chrome is important to Google. User behavior signals are really important to Google. Um I am doing the same things to improve user engagement on the website that I did before. Nothing's changed about what I did.

I'm just so happy that everything that I've been saying has been validated. And I had the biggest I told you so momentu uh of my SEO career. But I I kept it cool. I didn't want to rub it into anyone's face. I was just like, "Yep, let's just I'm I'm happy. I'm I was on the right track. I didn't waste anyone's time. I didn't burn clients money. We were doing the right things. The whole time."

So, it's like a super nice validation. That's that's what I take from the leak. Um, there's probably some good stuff in there that um that I could still play with. Um, but that's old gen. That's the That's the ball of yarn Google with a lot of band-aids on it. The mess the No, they call it artisal algorithm. Um, it's just a bundle of messy stuff that nobody at Google even understands.

I I think what Google is doing now, propelled by Open AI, they're they're moving full on AI. There's no stopping them now.

And I think that's the right way to go about it. Um, for better or worse.

Noah Learner

Here's a question. You could keep this answer. This could be a yes or no, but you can go into detail, too. Sure.

Are you cool with folks taking this transcript and having chat GPT create a few SOPs, frameworks, directions, etc. with it?

Dan Petrovic

Of course. Everything that I've ever said has been But but what I what I do request what I do request is that people give back. Yeah.

Noah Learner

Yeah.

Dan Petrovic

Don't just don't just like share it, take it take it, but share it, pay me with some cool ideas. That's the expectation. Like like I think one thing especially I I guess maybe we're like the older generation the younger generation keep talking about these like engagement user engagement social media LinkedIn I think in particular it's like oh my engagement you know audience like they're just like constantly hacking the algorithm trying to get views and and and trying to get like oh people not putting URLs anymore in their post they're putting them in the comments doing all these ridiculous things for engagement but

Why are you engaging? What's the outcome? What are you trying to do? Are you selling something? The only engagement that means anything to me is if you comment on my post with something critical or meaningful or contributing to the idea that I'm sharing. Exchange of ideas. Oh my god. Forums. We used to do that on forums. Remember that? What happened to that?

Noah Learner

I never joined warrior forum. That was, you know, early on I probably joined Maza's forum because

Dan Petrovic

I that was SEO something forum. There was an SEO I forgot what it was called.

Noah Learner

There's all kinds of them. SEO warriors. I don't I don't know.

Dan Petrovic

But I'm not talking just forums for SEO. I'm talking forums in general where people like

Noah Learner

Yeah. Yeah.

Dan Petrovic

Early days of the web, we would jump on and exchange ideas.

Noah Learner

For me was disc golf. I was in this disc golf forum where the disc golf this website um what was it called? I don't remember. It's like DG something where they had they had course maps for every disc golf course in the world pretty much and they had reviews of all the courses. So you could plan a trip like Amy's planning a trip to go see Wooden Trolls. For me it was like oh I'm going across the country. I'm going to play disc golf as I go across the country. Oh, dgcourse review.com. Yeah. So, yeah, forums were a thing and they still are. I mean, they drive a ton of traffic and a ton of visibility and, um, like the local search forum is a key competitive advantage for the agency that I work at, Sterling Sky. Um, so I know they even, you know, you talked about them historically, but they still matter.

Dan Petrovic

Um yeah, I mean Reddit's kind of taken over as a world world's forum for everything. Um but the point is do what you wish. Um but do contribute and share and and and um don't be shy like if I post something um jump on and exchange ideas. Um people have been some people be pretty cool like oh what is this? I don't understand this. Can we I've been jumping on calls with with some of the SEO people just like hey can you I'm really overwhelmed by this whole AI stuff um and we jump on a call and I you know um help them get on the right path.

It's been really really rewarding. Community has been great overall except for acronyms that really annoy me.

Noah Learner

Um Oh yeah. Yeah. I can't stand it either. Can I show you a prompt and maybe you could tear it apart?

Dan Petrovic

Sure.

Noah Learner

I think this might be fun for folks who are learning how to build props. Um, so I built this in AI studio and I pasted it into other models just to see how the results would be different. Um, I wanted to know about intent and I wanted to start with an entity and then break out to understand the intent and then I wanted to understand how the model was getting its source material. And this is probably too small for everybody. I don't know how to make it bigger. You're a world-class knowledge architect and user intent specialist. Your mission is to generate a comprehensive structured analysis of the entire user intent domain for a given entity grounded in live authoritative sources from the web. And I how do you like to start when you're building when you're building prompts? Like do you always start with role? Do you never use role?

Dan Petrovic

I I have a thing against getting models to let these very large models to do very simple things like uh querying intent uh for example. So um a prompt like this for me wouldn't actually exist at all.

Noah Learner

Tell me more. So I'll stop sharing.

Dan Petrovic

Uh so so basically what you just did in there would be a prototype for me. Yeah. To jam out some ideas with a very large model.

Noah Learner

Yep.

Dan Petrovic

Then what I do is I train a small model to do one thing. Y and one thing only and do it really well. And people like oh he's building machine learning models now. This is way above our you know like what we norm what we're supposed to do. But it's not I can't code Python. And there you go. I made a model. Yes, of course it does. It's a little bit of a learning curve, but the reward is there.

Um, so I'll give you the one recent example. Um, so what we did is inspired by Glina, the named entity recognition model where you can assign arbitrary labels. Um, I built a model that can basically take um arbitrary label schemes. So you can say um branded, non-branded, formational, local, um navigational, whatever. But then you have a specific you have finance industry and then you go um credit uh this and that.

So you can give it any any types of labels. Um and you give it a description as context to that label. So um my favorite absolute f favorite model to build on top of is Microsoft's Divera V3. So, the beta v version three and I think if you have even a modest GPU or if you if you want you can you can train these models on Google's cloud or whatever.

Um it's like a super powerful model that you can train in one day on what is it?

Noah Learner

What's the model called? I'll look it up

Dan Petrovic

It's called uh deberta v3.

Noah Learner

How do you spell that?

Dan Petrovic

D era.

Noah Learner

Okay.

Dan Petrovic

V3.

Noah Learner

Okay.

Dan Petrovic

Yeah, the Verto V3. Um, absolutely fantastic model. Great for f great for fine-tuning further for classification. Um, it's just an incredible little model. Um, and so basically what what I'd do is that prompt you had, I would get Gemini to Yeah, I would do that to create some training data for me.

So basically I get Gemini to create synthetic data set with already things labeled. Once I have that I train a tiny little model. So next time I'm not wasting money on API calls. I'm not doing anything um um gigantic. I have a tiny model that runs on my computer and I just go within one minute I've just classified 10,000 queries. And so that's the approach.

Noah Learner

So you mentioned earlier that you have learned all this stuff in three years. Take us through that learning journey like how could someone else get started on the path. How could someone do it efficiently?

Dan Petrovic

I guess would be the hardest the hardest question to answer. I guess good news is you don't need three years anymore. Three years to took because the models were really dumb at the start. Um, but now they're really capable and they're a lot better as a coach, a lot better as a guide. You don't need a uni degree. You don't need all these fancy courses or or sub subscribe to somebody's new newsletter. Like uh like some some things you need you need a setup. You need to install you need basic mastery of command line interface.

You need to install Python and you know like these these basic things and you can just say hey I want to get into this teach me tell me what are the steps and let it guide you um and if you if you don't know what you don't know let ask GPT or Gemini to interview you. People don't realize like interview me sus out what I know and I'll tell you like it's rather like you can't think of anything to say to it but if it's interviewing you for the purposes of skilling you up then it's going to have a lot better starting point and start with a start with a problem with the with the what are you trying to achieve? I've got a client um okay I'll tell you about the problem that I solved today that I didn't solve before. So we have a new client that we're on boarding right now. They're in like into camping gear and batteries and stuff like that. And um one of my teammates shared um um Google tonomy you know the Google tonomy file txt file that they shared you know like the tonomy ID and the tonomy pathways and all that.

So the client has 10,000 URLs. It's not too big. Um, so you could technically do that manually, but my god, 10,000 URLs, mapping, um, ids and all that. It's So, um, I was like, hey, how can I do this quicker? So, I'm like, I have a problem. Let's solve this. So, back and forth, back and forth. What we ended up doing, and this like I think people are going to love this and might take take and run with this. Generate vector embeddings for each line of the tonomy. Create a vector for each one. And of course keep the text associated with that.

And then do the same thing for the content of your pages that you have that you scraped or the titles of the page or meta description or whatever. And then by cosign similarity map the most likely category in the in that tonomy to the to the most likely page.

Boom. Like within five minutes like less than five minutes I had the entire e-commerce website mapped to the Google tonomy.

Noah Learner

Damn. I could have done that in two hours and I don't know what I'm doing. I'm saying like I I learned I built a um a recipe for Screaming Frog that they just put in their in their tool set to do page, excuse me, passage level embeddings. When you're doing this task, you're doing it at the page level, right?

Dan Petrovic

Um well, actually we have we have sentence level, we have page level, um we have meta level. So, we we sometimes work with as little as title, description, um, and maybe H1, but sometimes when the when the website is really big, so like, um, last week or two weeks ago, we worked with a website with 250 million pages.

Um, and I didn't feel like we it was it would be reasonable to scrape that site or even ask the dev team to export all that data. Um particularly because there's a lot of it is from your lake and uh generative like programmatic um and so what we did is we analyzed because the URLs are nice and descriptive so that's lucky if there are just random numbers that you can't do this but you can effectively what you can do is get the site maps download everything um store all the URLs in database like SQLite or something like that and then generate like extract the keywords from the URLs themselves put that into vector to space and then map things together. So basically u URL semantic semantic URL context and then mapping.

Noah Learner

So you just so like what you just said would make um Hamlet Batista smile from from the other side, right? I mean that was do you remember his challenge like what can you do with URLs and John Merch and Dave Sodto were doing this crazy stuff extracting meaning from URLs. That's amazing.

Dan Petrovic

And SEO community has embraced semantic uh search and vector embeddings to the point where it's become a gimmick.

Like every now and then it's like, hey, I'm using vector embeddings to do this and that. Like there's a lot of misconceptions. Um like I've I've made an offer to the community like if you're really like baffled about something and if you think you might be using things wrong, just let me know and I'll like try to point it in the right direction. People are I think romanticizing and giving semantic similarity cosine similarity over vector embeddings is a bit too much power. It's just one metric that you need to use with common sense and in combination with all the other u things that you have including huristics.

Um that was a great idea that popped up. Let's do site migration based on vector embeddings from old URLs to the new ones. When I tried to actually implement that in practice, it didn't work for many many reason I I'm not going to go into but like what it ended up being ended up being an aiding factor rather than the deciding factor. It was like a improvement recommendation improvement score. Um yeah, but I feel like we've slightly diverged from the last uh last question, you know, about the prompt.

Did did we wrap that up? I I suppose yes small models but I think the point I'm trying to make is small models are cool don't use Gemini if if you can have a small powerful model that does one thing and one thing only and does it super well um I'll I'll try to make a strong case for this um encourage everyone to try to do this.

I have a link model called linkird and linkird can recognize

Noah Learner

Played with that last night yeah

Dan Petrovic

in in plain text it can spot when something feels like it should be a link. Uh and if you I've done massive amount of analysis on that and we have a version of link bird that's internal a lot lot bigger model and it makes linked predictions better than Gemini and Gemini is way better at everything else but link is better than Gemini at links. So that's the point I'm making. Small tiny model that can run on your computer that only does one thing can do that one thing better than a a major major model. That's that's a really strong statement. I stand by it.

Noah Learner

I love it. So, um what did I what did I miss, Dan? What didn't I ask you that you wish I asked you?

Dan Petrovic

Ah, we could we could talk for um for hours. I think uh we've covered covered enough ground to get the conversation going but like I think what what needs to happen is like people who are interested in this and engaged need to really be engaged and start start the conversation and discussing these types of things ping me in comments.

Hey Dan, what do you think about this? Get get this happening because the the message I'm sending out to everyone is like we no longer have to pray to the gods of AI the way we used to pray to the gods of Google because we have a lot more insight and a lot more u ability to interpret all this. It's not easy. I'm not I'm not saying it's easy and I'm not saying it's necessary. You can continue like rolling with your usual SEO stuff as you did before. Um, but I think a lot of people are trying to make sense of AI and I'm saying there is a technical aspect of it. You have to decide which way you're going.

Are you going to be like a content type influencer for AI hoping for the best or do you want to really push the buttons if you choose if you choose the blue pill or the red pill? You got to decide. Um but if you go down down down the track of you know like I really want to understand the models and how they work and what makes them tick then you have to like I think you have to get into some level of technical expertise. It was hard it was foggy. It took three years like I said it was very difficult but I think you and I a little bit similar. You mentioned p sheer persistence.

Noah Learner

Yeah.

Dan Petrovic

Um and determination that's that's a quality that will help you.

Noah Learner

Um, I got started with raves, too, by the way. Yeah, but it was me designing party flyers back in 2001, two, and three for I can see them in my head.

Dan Petrovic

I can see them. I can see the flyers. I know the color schemes, the elements, the 3D stuff.

Noah Learner

Yeah. Uh, and um, so when you when you mentioned that, I wanted to ask you what groups you were into back in the day. Like for me it was Scrance and Spongle and stuff like that.

Dan Petrovic

Definitely Scitrance. I still I still uh play some good side trans every now and every now. Um Frana Frana Geomantic um album and DOF.

Noah Learner

I'm gonna see Infected Mushroom and

Dan Petrovic

Oh, they're in Australia this week, I think.

Noah Learner

Yeah. Uh I saw Spangle. Well, actually it was just Simon Poser, but I saw him do two nights in Denver in April and it was just an amazing show. I was front row center and uh it was after listening to them for 25 years or whatever. It was pretty amazing to see him.

Dan Petrovic

Two hours before joining this this uh call with you, I was at Kiasmos the um Icelandic um electronic music performance right here in Brisbane. I literally just returned from one. It's been a while since I've been to one. This is This is a bit more like a sitting environment.

Noah Learner

Yeah. Well, I just got to remind everybody it is 1:05 a.m. for Dan and somehow he's still lucid. Amazing. So, uh we should we should probably wrap this up, but I'm hoping that this is the beginning of a beautiful relationship for us and that we can continue the dialogue um online and hopefully jump in some Zooms uh for everybody. Um in the audience. I'm so glad that you shared this time with us. I I know I didn't get to all the questions. I I tried to get to some. Uh Dan, keep it up. I think the the question that I'm left with is uh like what is my learning path? And I think people need to ask themselves that question. And then balancing how much time and energy I'm spending when I'm looking at my data and I'm still seeing that traffic from these sources to my website might make up you know onetenth of 1% of all the traffic. So, it's like how do I how do I manage that time investment? And is it am I going to do just in time learning or am I going to do learning in advance so that I'm well positioned and have like years of competitive advantage over everybody else? Like it's kind of how I'm thinking about it. Yeah, pretty wild.

Uh what parting thoughts? Uh any soap box? Anything you want to um share with the audience that's outside of SEO that you think can make the world a better place? Anything that you want to share?

Dan Petrovic

I think um um one thing that um I've been trying to implement um because of what's been happening in our industry lately, I've I've been burning hard for 3 years now. Um and I noticed a lot of stress and anxiety with me and my team. And I just reminded everyone there's always another day. Um like just abandon your work 5:00 just leave everything have a life. Um go for a walk do the thing that you want to do come back to tomorrow it's waiting for you there. Um don't tell your client it'll be ready next week tell two weeks from now um if you if you finish it early you'll delight them. I mean this these are simple things that we know everyone knows this but we're not respecting our time and we're not respecting ourselves.

And so at a time like this when everyone's trying to skill up, get like oh like this crazy guy just spent three years studying machine learning that took a toll. Um that didn't come for free. Um and I decided, okay, I got to draw draw the line. I'm like I I've pushed myself physically, mentally as far as I could. Now it's time for self-care and looking after myself. And I've been sort of preaching the same thing to my guys in the team and I've been saying just leave it. It'll come like you get back to tomorrow, you'll be nice and fresh.

Um so that's something that um I've I've always known about this and I've been bad and not respecting it because addiction, you know, everything is so interesting and there's not enough time.

Noah Learner

This hits so hard for me because that was my 2019 to 2023 life was just constant like my brain was on fire and I was so emotionally invested in all of that innovation learning and I went from doing like Google Sheets formulas to building robust data pipelines and the amount of learning to get to that place and building stuff that was item potent meaning, you know, you run it on Monday, you'll get the exact same outcome on Tuesday and Wednesday. That uh I burned out and um it took me a long time to get back to a place where I'm even considering doing like a huge big push again. So, it's uh it's what you're saying resonates in a big way.

Dan Petrovic

Yeah. Look after yourself.

Noah Learner

Yeah, 100%. And and I've been doing a lot of exercise and I'm riding a bunch and I even started uh an SEO community straa club just yesterday because I was like, you know what, this is the best form of like peer pressure possible is getting someone to want to like go exercise a little bit and not be a maniac. But it's like, man, should I get out of the house today or not? It's like, yeah, because my friends give me kudos. Would I go on a bike ride? That's so cool. So yeah, I'm all in on

Dan Petrovic

We should do a rave. We should do a rave. Yeah, right.

Noah Learner

Yeah. Oh, it's funny. Um, love it. Uh, talk to Lily. I'm sure she's in.

Um, excellent. Well, Dan, this is amazing. So great we got to meet. So great we got to spend an hour and 10 minutes together. Um, I uh this is now live on YouTube. And for everybody else at home, thanks so much.

I just want to thank our sponsors again. Hrefs, Aerops, Citation Labs, Drivefly Digital, Demand Sphere, Bendo, ModX, SEO Jobs, and our brand new community partner, Jet Octopus, which I'm so grateful for. Um, we'll be in touch.

Everybody, thanks so much. This has been another amazing EP episode of Campfire Chat. Dan, thank you. Thank you. Thank you.

Dan Petrovic

You’re welcome

Noah Learner

And stay in the room because it'll take a minute for this to upload. But I'm gonna stop the recording and thanks so much everybody. See you soon. Thanks.