Evaluating content with AI on a large scale - I have an app for that
Recipe:
Google content guidelines + advertools + OpenAI API + Dash
???? Content guidelines: Set of questions to ask about a web page’s content.
???? advertools: crawls the given pages, and extracts the title and body text.
???? OpenAI API: Evaluates the content by answering each of the questions True/False.
???? Dash: A no-code GUI for using all this.
The summary is the average score for each question for a bunch of URLs.
Todo: Getting a “True” for a question is not necessarily a good thing, and it depends on the phrasing of the question. Answers need to be normalized, True = good, False = bad.
If you have traffic data about the pages, even better. You can do the evaluation weighted by traffic.
You can play with the app if you want:
Would love any suggestions, or if you get any issues.
Thanks!