Cookie Consent by Free Privacy Policy Generator

The Best Of

Go to the Best Of the SEO Community.

Noah
Noah
Jan 18, 2025, 9:05 PM
Forwarded from another channel:
Forwarded thread from another channel:
Brent Rangen
Brent Rangen
Dec 27, 2024, 7:36 AM
Interesting idea: anyone think it will get more traction?
A proposal to standardise on using an /llms.txt file to provide information to help LLMs use a website at inference time.
llms-txt: The /llms.txt file – llms-txt
John Mueller
John Mueller
Dec 27, 2024, 8:20 AM
This seems like a kludge. Just make a normal, crawlable site?
Dave Smart
Dave Smart
Dec 27, 2024, 8:39 AM
Feels like a real good way to feed llms one thing and users another too i.m.h.o. I'd also worry about synchronicity too, update / change / add to your content, easy to forget the llm version (although you could I guess automate that to some extent)
Brent Rangen
Brent Rangen
Dec 27, 2024, 10:04 AM
Definitely not setting the internet ablaze. ????
MIchael Buckbee
MIchael Buckbee
Dec 30, 2024, 6:02 AM
Couple different things:
• We have quite sophisticated tools for taking HTML+Images and making it plain text of all sorts, so I'm not sure it helps anything there.
• It's built on the assumption that LLMs just want text, but the newest models are quite mutli-modal and support images, video, etc. which makes this feel outdated.
• One larger strategic change is that you want more `Brand Statement` content as part of your overall content strategy than preAI and that might not fit cleanly into how you present your marketing, so it might be helpful for that.
Emina Demiri-Watson
Emina Demiri-Watson
Jan 4, 2025, 2:47 AM
Cue consultants blaming it’s absence for lower results.
MIchael Buckbee
MIchael Buckbee
Jan 4, 2025, 9:47 AM
@Emina Demiri-Watson well, I wonder how you even adopt it? Like do you re-render all your pages as markdown...that seems excessive.
Emina Demiri-Watson
Emina Demiri-Watson
Jan 5, 2025, 2:29 AM
@MIchael Buckbee surely not! I just think it’s an overkill but then maybe Im not understanding something here.
John Mueller
John Mueller
Jan 5, 2025, 3:26 AM
Random data point - I crawled all our documentation and converted it into markdown to feed into NotebookLM. It's not super-hard (I used colab). Anyone training an AI model can do this if they're already crawling the web or ingesting HTML content. (That said, it was pretty cool to have all the content in a single folder :-))
Emina Demiri-Watson
Emina Demiri-Watson
Jan 5, 2025, 3:51 AM
@John Mueller plus now you can listen to the worlds longest podcast!
MIchael Buckbee
MIchael Buckbee
Jan 6, 2025, 7:11 AM
I can definitely see there being "Packages" of some kind of documentation like this built for ingestion into your AI model of choice.
Andrea Volpini
Andrea Volpini
Jan 15, 2025, 1:38 PM
If you work with LLMs, markdown helps - this is why exists and service like are growing fast.
The best model to turn HTML to markdown in my personal experience is (ReaderLM-v2 is available from today)
A proposal to standardise on using an /llms.txt file to provide information to help LLMs use a website at inference time.
llms-txt: The /llms.txt file – llms-txt
Tavily is a search engine tailored for AI agents, delivering real-time, accurate results, intelligent query suggestions, and in-depth research capabilities.
Tavily
Reader-LM-0.5B and Reader-LM-1.5B are two novel small language models inspired by Jina Reader, designed to convert raw, noisy HTML from the open web into clean markdown.
Reader-LM: Small Language Models for Cleaning and Converting HTML to Markdown

Our Values

What we believe in

Building friendships

Kindness

Giving

Elevating others

Creating Signal

Discussing ideas respectfully

What has no home here

Diminishing others

Gatekeeping

Taking without giving back

Spamming others

Arguing

Selling links and guest posts


Sign up for our Newsletter

Join our mailing list for updates

By signing up, you agree to our Privacy Policy and Terms of Service. We may send you occasional newsletters and promotional emails about our products and services. You can opt-out at any time.

Apply now to join our amazing community.

Powered by MODXModx Logo
the blazing fast + secure open source CMS.