AI Detectors Flag Human Writers as Editors Screen Submissions
NPR reports writers falsely flagged as AI-generated by imperfect detection tools editors use to sort submissions, as a June study finds detectors disagree.
The story in focus
NPR reports that writers are being falsely flagged as AI-generated by imperfect detection tools some editors use to screen submissions. A June study in the International Journal for Educational Integrity found widely used detectors disagree and vary in accuracy, while magazines face rising submission volumes.
- Ghostwriter Sally McGraw says a client's detector results flagged her self-written sample as machine-generated
- Science blogger Janelle Shane says a detector gave 98.7% confidence that a passage from her 2019 book was machine-made
- A June study in the International Journal for Educational Integrity compared four detectors and found accuracy varied
- Clarkesworld publisher Neil Clarke says submissions doubled between 2021 and February 2023, prompting a temporary portal closure
- The Authors Guild said in May that detector use creates an environment reminiscent of witch hunts
False positives reach working writers
NPR reports that ghostwriter Sally McGraw submitted a writing sample for a book proposal job and was told multiple AI detection tools had flagged her work as machine-generated. The client confirmed screening the sample but said experience and fit determined the hiring decision. McGraw says she was paid for the samples but believes the results cost her the proposal work.
Science blogger Janelle Shane tested excerpts from her 2019 book and says one detector reported 98.7% confidence the passage was machine-generated. Chris Callison-Burch, a University of Pennsylvania professor, told NPR that detectors often work by asking a language model how predictable a passage is, and that models tend to find their own training text unusually predictable.
Detectors disagree and editors face volume
A study published in June in the International Journal for Educational Integrity tested four detectors on 160 academic papers spanning human-written, AI-written, combined and human-rewritten AI text. NPR reports Pangram was the most consistently accurate, correctly identifying AI use in 37 of 40 papers in the combined category and 37 of 40 in the human-rewritten category. Pangram CEO Max Spero said in a statement that the tool can err, especially on short texts.
Clarkesworld publisher Neil Clarke told NPR the magazine received about 1,100 submissions a month in 2021, a figure that doubled by February 2023 and led him to temporarily close the portal. He estimates his workload rose about 30% and says he does not personally use detectors, cautioning that submissions should end in rejection, not accusation.
What the flags mean for writers
The Authors Guild said in May that detector use creates an environment reminiscent of witch hunts, warning false positives can cost contracts and reputation. McGraw says she now avoids certain words and sentence constructions, including em dashes, to make her prose appear less machine-like. Callison-Burch said detectors weigh patterns across a whole piece, so a single word or punctuation mark does not prove AI use.
Do AI writing detectors reliably identify machine-generated text?
No. A June study in the International Journal for Educational Integrity found four widely used detectors varied in accuracy, and NPR reports documented false positives on human-written work, including a 98.7% confidence flag on a 2019 book excerpt.
Sources & further reading
Help keep the record accurate. Send a correction.
