How Do LLMs Process and Understand Images Online?

TL;DR

No clear consensus on how LLMs handle image accessibility markup - It's uncertain whether LLMs use HTML attributes like alt, longdesc, <figure>, and <figcaption> or if they perform their own image analysis through OCR and vision models, ignoring HTML annotations entirely.
Alt text likely matters for retrieval, but complex images are unclear - There's agreement that alt text helps LLMs retrieve information without expensive image processing, but it's unknown how LLMs handle complex imagery like infographics, data sheets, or scanned documents that traditionally require detailed accessibility descriptions.
This is a rapidly evolving area that needs testing - The technology is moving so fast that answers become outdated quickly, and agentic browsing adds another layer of complexity; specific testing is needed for different use cases like product images, infographics, and data sheets.

Jarno van Driel

Nov 14, 2025, 2:33 PM

Any of you know of any (scientific) publications about how LLMs (and tech stacks connected to those) handle/digest images on web pages (during training but also by means of RAG), specifically if/how/which info they extract from images (is that limited to certain verticals like shopping, or done for the full corpus of the web)?

I had a conversation yesterday with somebody who turns statistical information into images (e.g. infographics) for a living, who told me about how much time he daily spends on making sure the content of such images is properly expressed by following WCAG (accessibility) guidelines. A very interesting conversation, but at the end of it we were both left wondering how much of that holds up in the day and age of LLMs.

Do LLMs even take attributes like `alt` and `longdesc` , or tags like `

` and `

` into account, or do (certain) LLMs perform their own image analysis (ignoring the `HTML` annotations completely), should we now extensively annotate images through structured data markup, etc.

I really don’t know what the current state of affairs is, and I’d love to learn more about that.

CC: @user, @user , @user

???? 3

Tory Gray

Nov 14, 2025, 3:02 PM

Oh fun - I’ve been digging into similar things - so following along here. And, not a journal, but one piece I did like was https://www.neurospicy.agency/post/multi-modal-images-context-engineering re: implications of physical product packaging and those images

❤️ 1

Jarno van Driel

Nov 14, 2025, 3:12 PM

Thanks @user, that was a nice start.

I was aware of OCR already (and I believe specialized vision LLMs are a thing as well), but some of the things I’m wondering about are: How much is it actually used (as in, is it used for every image on the web, or only specific ones). How much of the info contained in an image gets extracted for which use cases (I know, that’s a moving target. These techs evolve really fast).

A product image is one thing, but how about large infographics, scanned pages, or even tables dynamically rendered as images (I know, bweegh, but alas it’s quite normal for that to happen).

???? 1

John Mueller

Nov 14, 2025, 3:44 PM

There's a lot to this, some of which you can test. But one interesting development - which also clashes with the current JS-thinking for LLMs - is agentic browsing.

:100: 1

Jarno van Driel

Nov 14, 2025, 3:58 PM

There’s absolutely a lot to this @user, and I’m aware that it’s near impossible to give any straight answer. Not only does every answer lead to new questions, but also the development of the techs involved is moving at such a ridiculous velocity that anything one says today, can be outdated tomorrow.

That’s why I’m trying to focus my questions to the actual application side of things, while trying to avoid questions about what is possible. For the latter, Google Scholar offers plenty of rabbit holes to get myself lost in.

Right now I’m mostly curious about whether or not image accessibility guidelines actually matter for LLMs or not (after all, there’s a big difference between good-ol’ screen readers and LLMs, although that gap is closing fast). And if not, than what does matter for LLMs? (I know, there are many sorts of those)

Andrea Volpini

Nov 14, 2025, 4:00 PM

I believe it’s fair to assume that adding alt title on images makes it easier for the retrieval layer of LLMs to grasps insights from images (without any expensive image processing)

???? 1 :heavy_check_mark: 2

Jarno van Driel

Nov 14, 2025, 4:14 PM

I agree with you on that @user, though when it comes to accessibility there’s more to it than just providing an `alt` text (although I consider that to be an art in itself). Especially with complex imagery containing a lot of info (happens a lot in governmental and institutional domains). That’s where for accessibility `longdesc` comes into play (especially for images that are meant for redistribution).

Something that can be used through screen readers (I don’t believe it’s being used for search), but question is, do LLMs take that into account, and if not, do they try to analyse complex images, like infographics, or do they keep it at the `alt` text provided?

Andrea Volpini

Nov 14, 2025, 4:15 PM

absolutely - this is an interesting test to make

:100: 1

Andrea Volpini

Nov 14, 2025, 4:15 PM

on retrieval I believe they don’t - but worth testing - on upload they might

???? 1

Jarno van Driel

Nov 14, 2025, 4:54 PM

To make a test possibly applicable to Wordlift @user… Product data sheets, something manufacturers often provide as either an image or pdf. I can image GS1 (Italy) being interested in that as well.

Victor M Pan

Nov 17, 2025, 2:21 PM

Here's what I know:
Training = Datasets. You're better off looking at what LLM's are trained to do in OCR.

I'd infer text in image is recognizable in Gemini, for example because there's training data in there: https://huggingface.co/datasets/google/imageinwords

The only other person nerding out on this that I know of is Dan P. :joy:

I don't know about longdesc, figure, and figcaption - but at web scale I think alt would be the most sensible one to tackle.

???? 1

Victor M Pan

Nov 17, 2025, 2:34 PM

In terms of tool use to prompt, tools within the API and/or tools referenced in system prompts give us a sense of what "skills" each LLM is able to use to tackle a user's request.

I don't specialize in image search at all, so I'll also be following along - I'm sure the folx who work in stock imagery would have a lot to say about this subject. Perhaps reach out to those SEOs?

Jarno van Driel

Nov 17, 2025, 2:56 PM

> “Perhaps reach out to those SEOs?”
Good call, maybe @user has something to add here.

Thanks for your comments though @user, and agreed, Dan P is a serious nerd indeed, hehe. Too bad he isn’t part of this community. Maybe @user can nudge him in this direction.

???? 1

Channels

agency (3)

ai-for-seo (11)

ai-llms (24)

analytics (13)

biz-of-seo (4)

career (4)

content (15)

cro (4)

data-viz-reporting (4)

datascience (2)

ecommerce (3)

events (1)

freelance (2)

general-seo (21)

in-house (2)

local-seo (12)

local-seo,biz-of-seo (1)

lounge (1)

reading-list-resources (3)

search-console (11)

shameless-plug (1)

technical-seo (37)

tools (1)

Join The Community

The SEO Community is a free organization, with private Slack group and events, for those interested in search and AI optimization to learn, discuss, and meet others in the industry.

How Do LLMs Process and Understand Images Online?

TL;DR

Channels

Join The Community

Resources

The SEO Community © 2026

How Do LLMs Process and Understand Images Online?

TL;DR

Channels

Sign up to get the newsletter

Join The Community

Resources

The SEO Community © 2026