Unlocking Relevance: Vector Embeddings & Cosine Similarity in Content Search

The Best Of

Nov 12, 2024, 5:33 PM

How are people thinking about vector embeddings and cosine similarity to understand how content is relevant against search queries?

Google is using these types of calculations across the very piece of text on the page. As AI Overviews begin to propagate across SERPs it seems like this will be increasingly important to understand…

Forwarded thread from another channel:

James Gibbons

Nov 12, 2024, 5:36 PM

AI Overview tracking is brand new; we can now understand when the appear and who may be cited… understanding which parts of text Google is showing and the delta of those scores compared to a client’s content may be key in ‘optimizing’ for AI overview as well.

Shawn Huber

Nov 12, 2024, 6:07 PM

Chris Long wrote a pretty solid article on this:

Everett Sizemore wrote one on embeddings and internal linking opportunities - this one I've started using on a few small side clients. It is too early to say if there have been any changes as a result of using it, but either way, it does highlight some really awesome internal linking suggestions.

James Gibbons

Nov 12, 2024, 6:28 PM

I saw Chris Long’s article and was spot on, forgot to include it here and thanks for sharing Everett’s…. Mike King’s talk at Tech SEO Connect was great as well showing the gap in the industry

James Gibbons

Nov 12, 2024, 6:29 PM

Michael King - Accounting for Gaps in SEO Software

Deck by

Tech SEO Connect

Speaker Deck: Michael King - Accounting for Gaps in SEO Software

Shawn Huber

Nov 12, 2024, 6:33 PM

And the latest release from ScreamingFrog will make automating this a lot easier too

Valentin Pletzer

Nov 12, 2024, 10:55 PM

I believe it is super fascinating to understand embeddings and their potential usecases in SEO. That is why I created

That said: we don’t know which embeddings Google uses and probably never will. And in all likelihood the selection criteria of AIOs are more than just cosine similarities

simcheck - Chrome Web Store

a chrome extension to generate, compare and visualize vector embeddings

James Gibbons

Nov 13, 2024, 8:25 AM

@pletzer whats the main use case for the extension? is there a workflow you had in mind? I think its interesting to consider what is the 'source/target' of the comparison...is it internal content, keywords to content, internal content to competitors etc

Valentin Pletzer

Nov 13, 2024, 8:29 AM

I built it first and foremost to give people an easy possibility to create and experiment with vector embeddings. But I have used it to compare two sets of content and find near duplicates as well as shared topics. I documented a bit here:

James Gibbons

Nov 13, 2024, 8:41 AM

Cool I’ll take a look

Robin Allenson

Nov 13, 2024, 10:58 AM

Some things to consider: not all embeddings are created equal. They depend on the model and what it’s built for. Often they do well on longer text, but less well on very short text _e.g._ keywords. Before the foundation model era (“before AI was AI” someone said to me), we trained our own language model to better understand eCommerce keyword data. See . That taught us a laundry list of what not to do (but luckily we had a lot of fun and the model was super useful).

One of those was — although foundation model embeddings like understand more nuance, they may still lack the eCommerce vocabulary for longer tail queries. They can also be slow because they are such long strings of numbers. You may do better with smaller models that handle out-of-vocabulary words better _e.g._ by using substring embeddings. That means that could sum the vectors for the component 3-character substrings that make up a word.

AI for SEO: learn which keywords are similar, without labelling data - Similar.ai

Intent classification Product-led SEO teams are often interested in how to use AI for SEO, and many are enthralled by intent classification. Intent uses AI to understand how similar the meaning of two keywords is. Specifically, intent classification is a kind of AI tool to understand the implicit expectations about the types of pages which

Written by

robin

Est. reading time

13 minutes

OpenAI Platform

Explore developer resources, tutorials, API docs, and dynamic examples to get the most out of OpenAI's platform.

Valentin Pletzer

Nov 13, 2024, 11:58 AM

That’s why the extension allows for downloading different modes from HuggingFace

Ray Grieselhuber

Nov 13, 2024, 12:57 PM

We're integrating them into our platform, starting with our AIO tools but will be adding elsewhere as well. It's a useful metric to help people understand inclusion (although I don't think it is or will be the be-all, end-all).

James Gibbons

Nov 13, 2024, 1:03 PM

Totally agree that not all embeddings are created equal, they can also be weighted by relationship to visibility in the SERP, we've (Quattr) leveraged GSC clusters and competing urls create a workflow for relevance scores with the embeddings across all aspects of a given set of ranking pages in a given keyword cluster