When you're using LLM for search (either for yourself or client wor...

The Best Of

Go to the Best Of the SEO Community.

Noah

Jan 19, 2025, 10:31 AM

Forwarded thread from another channel:

Mika Lepistö

Jan 9, 2025, 6:55 AM

When you're using LLM for search (either for yourself or client work) are you asking it to provide details on the data sources and search terms it's using?

Wondering if this is a way to help log and understand the search choices its making.

Jan 9, 2025, 6:59 AM

Sometimes I asked it to cite sources but not all the time, case by case.

Andy Strager

Jan 9, 2025, 7:02 AM

When I am doing content briefs I always ask for it to cite sources. If you have a good writer that goes the extra step to read the sources it really changes the quality IMO

Mika Lepistö

Jan 9, 2025, 7:04 AM

I meant more like this to understand what it's doing, for the potential to be able to influence it in the future. For simple searches it seems to just use the same search query, but for more complex ones there are often a few different terms.

For input "best accounting software for freelancers"

After it's done:

"Show me the exact search terms you used if you searched the internet. Also list other data sources you may have used and details on what queries you used with them. "

Search Details:

Search Terms Used:

• “Best accounting software for freelancers”

Source:

• Search engine query, accessing a variety of web-based articles and reviews from sites like:

• The CFO Club

• Zapier

• FitSmallBusiness

•

Data Sources and Queries:

a. Web Search (via Search Tool):

• Query: “Best accounting software for freelancers”

• Purpose: To gather up-to-date, relevant, and expert recommendations.

a. Internal Knowledge Base:

• Context: Recommendations for accounting software based on pre-2023 knowledge.

• Tools: Previous data on FreshBooks, QuickBooks, Wave, Zoho Books, and Xero.

No additional tools or external APIs were used in this instance. If you require further breakdown or alternative queries, please specify.

Andy Strager

Jan 9, 2025, 7:07 AM

Gotcha, no I haven't but that is a good idea. I may or may not steal this prompt sequence ????. I could see myself using this a few times a day but only for more in-depth stuff.

Mika Lepistö

Jan 9, 2025, 7:12 AM

Steal it! Let me know if you see any patterns.

I'd be interested in learning:

• frequency of how often it uses a materially similar approach for the same query

• frequency of the same sources for the same query

Eoin Clancy

Jan 9, 2025, 8:40 AM

I have a diff use-case, but have asked perplexity to do something similar before. With that said, how they are conducting their research will be different to how a search engine actually works today.

Mika Lepistö

Jan 9, 2025, 9:21 AM

Yes. Have you explored that further? I find it interesting to review. Credit to @Matt McGee for this approach.

Like this prompt:

After performing the web search what did you do to evaluate and respond. Give me a detailed step by step answer.

That was an aha moment in terms of it telling me it referenced memory about me on some things, so individually controllable personalization is going to be a factor.

Anne Hennegar

Jan 9, 2025, 9:33 AM

While it won't give you everything you asked for, have you tried Gemini Advanced 1.5 Pro with Deep Research? It clearly lists the websites it researched and provides clickable links. It also cites the sources within the doc it creates.

Before it begins, it asks you to approve the plan it provided.

Ryan Jones

Jan 9, 2025, 1:26 PM

unless it's doing RAG it doesn't know it's sources. LLMs don't work like that. they work on predicting which word comes next - so even if it tells you sources, it's just the most likely words that came next - not the actual sources.

Mika Lepistö

Jan 9, 2025, 3:27 PM

I do understand that, but it seems to correspond to the sources listed in the "show sources" that opens the citations if a web search was performed.