14
Last Tuesday my AI content detector flagged a human-written poem as 94% AI generated
I was running some tests on one of those AI detection tools for a client project, you know, the kind that schools are using to catch students using ChatGPT. So I grabbed a random poem from a 1987 issue of The New Yorker just to see what would happen. Robert Frost stuff, classic. The detector said it was almost definitely written by an AI. This is the same tool a university near me in Austin is using to decide if kids are cheating. I ran it three more times with different poems from different decades, same result. It flagged Shakespeare too. Something is really wrong with how these things work. Has anyone else tested these detectors with old human writing and seen the same thing?
3 comments
Log in to join the discussion
Log In3 Comments
joel_martinez1mo ago
Read a study saying these detectors fail 30% of the time in controlled tests.
4
the_lee1mo ago
Oh yeah I tested the exact same thing with a copy of The Raven and it pegged Poe as 89% AI generated. These detectors are basically just looking for patterns they think are "robotic" when really good writers just use clean language.
3