ABOUT THIS EPISODE
Last week, Substack released a new feature to detect AI-written text. They partnered with Pangram to let readers scan a post and see how much of it is AI-generated. I both love and hate this update, and I want to share my views with you.As I read it, they’re attempting to solve for two problems: the rise of AI slop, and reader trust. Both are real concerns. However, there is a big difference between AI slop and AI-enhanced posts, and Substack AI score does not make a distinction between those two.With that said, Substack is upfront that Pangram isn’t perfect, just that “independent research suggests it detects AI-generated text with a high degree of accuracy.” Fine.But they also admit the actual limitation:“Pangram can only detect whether AI was used to make the text, not whether great human care went into creating it, nor whether AI tools were used as a source.”Anyone who’s spent time in higher education already knows this. Detection tools flag patterns, not quality, and not authorship in any meaningful sense. Moreover, there are good and bad uses of AI, and that nuance is lost in the Substack AI score. The problem I have with this is that the score measures how much AI touched the writing and editing, not how much AI shaped the thinking, whether it was used for research, or whether a human sat with the final draft and meant every word. The Substack measure doesn’t capture when AI is used for the followingIdeationResearch and sourcesImages and visualsIt strictly captures the use of AI in text. Read more at DecodeEcon.com about what the research says will happen.
