--- title: Readers Cannot Tell Whether a Short Story Came From a Person or ChatGPT description: A Villanova University study found readers guessed the author of a short story no better than chance, and rated stories higher when told a human wrote them. author: Dr Marina Nani (Editor-in-Chief) date: 2026-08-05T15:27:50.930Z updated: 2026-08-05T15:33:18.297Z canonical: https://richbooksmagazine.com/article/readers-cannot-tell-ai-written-short-stories image: https://cdn.nanimediahouse.com/ai-written-short-stories-study-103412.webp categories: Authors & Writers content_type: News region: Global publication: Rich Books schema_type: Article --- Readers asked to say which of two short stories a person wrote guessed no better than a coin toss, and gave their highest ratings of all to stories they had been told a human wrote when ChatGPT had in fact written them. The finding comes from a [study published on August 4 in the journal Judgment and Decision Making](https://www.cambridge.org/core/journals/judgment-and-decision-making/article/bot-or-not-can-people-tell-the-difference-between-stories-written-by-a-human-or-by-an-ai-system/45E6DC0BB90AA648654D5AE243F6C667), by Sydney Sears and Deena Skolnick Weisberg of Villanova University. They gave adults aged 18 to 81 three short stories by human authors, taken from a literary journal and a short story collection, alongside three stories ChatGPT generated on similar themes. ## Told a Human Wrote It, the Same Story Scored Higher In the first experiment, 1,682 adults recruited through the research platform Prolific each read one story and were told, correctly or incorrectly, whether a person or ChatGPT had written it. They then rated its quality and how absorbing they found it, a measure covering attention, emotional engagement and mental imagery. The AI-generated stories came out ahead on both counts, by 6 percent on quality and 8 percent on engagement, as TIME reported, and the human label added roughly 3 percent on top of whatever a reader had in front of them. The highest scores went to AI-generated stories that readers believed a person had written. "In our study, participants gave the highest ratings to stories that they had been told were written by humans but which were actually written by AI," Deena said. "I'd argue that this reveals a bias towards narratives written by real people. We assume creative writing requires uniquely human qualities, such as emotional understanding and lived experience, which leads people to underestimate AI's capabilities." ## Readers Guessed the Author No Better Than Chance Two further experiments, with 424 and 481 people, gave each reader one human-written and one AI-generated story on the same theme, said nothing about where either came from, and asked which was which. In the second experiment 39.93 percent guessed correctly, worse than random guessing would produce. In the third, 51.97 percent did, which is no different from chance. "There is no general evidence that participants were able to distinguish human-written from AI-generated stories," Sydney and Deena wrote. Their own summary of the three experiments is that AI systems can produce creative work seen as at least as good as human work, while readers still do not believe they are capable of it. ## Familiarity With AI Tools Predicted a Correct Guess Self-reported expertise in fiction made no difference to who guessed correctly. Familiarity with AI tools did. In the second experiment, each additional point of self-reported AI expertise raised the odds of a correct guess by 14 percent. In the third, which used a validated Artificial Intelligence Literacy Scale, a point raised them by 33 percent. Deena put that down to pattern recognition rather than any special insight into fiction, naming em dashes and sentence constructions such as "it's not just X, it's Y" as the giveaways readers learn to see. "AI writing has a particular style that we can learn to recognise with practice, just as we can learn to recognise different human authors' unique styles," she said. ## The AI Stories Stated Their Themes Outright Asked why the machine-written versions scored well, Deena pointed to how they handle meaning. AI prose is clearer and easier to process, she said, while "human-written stories are often more subtle and complex". "For example, the AI versions of our stories usually stated their themes explicitly, rather than allowing readers to infer meaning from the characters' words and actions." > "AI writing tends to be clearer, more direct and easier to process." > — Deena Skolnick Weisberg, Villanova University Cameron Jones, an assistant professor of psychology at Stony Brook University who was not involved in the study, told TIME the result fits years of research on how poorly people distinguish AI from humans, including his own work on conversations where a person can question the model directly. He traces the appeal to how the models are trained. "We basically get the model to generate little bits of text, and then we get people to read them, and they give a thumbs up if they like what the model's saying and they give a thumbs down if they don't like what the model's saying," he said. "Real humans are weird and idiosyncratic, and they're very different from one another. But models can kind of learn to be this kind of milquetoast everyman who appeals to everybody." That, he said, can flatten the writing while lifting its broad appeal, where a human might produce something exceptional that suits only some tastes. ## The Study's Senior Author Writes Fiction Herself Deena does not read the result as a case for handing fiction over. "We may need to make room for AI-generated novels, and for AI/human co-written novels, but that doesn't mean that there's no longer space for us to appreciate the process of human creativity," she said. "Humans write as a form of creative self-expression or to challenge ourselves, or to make sense of our experiences. The fact that AI can generate human-like stories doesn't change any of that." Luke Kennard, professor of film and creative writing at the University of Birmingham, who was not involved in the work, takes a harder line on what the technology is. "Because we're human, we tend to personify AI and imagine it as some kind of glowing robot with a benign and neutral face, when it's really just predictive code based on a massive stolen database of actual writing, physically based in vast, expensive and destructive datacentres with catastrophic repercussions for the surrounding communities," he said. He argues it is not a tool in the way spellcheck or a thesaurus is, but an existential threat. That argument has a recent history the paper itself cites: the fear that AI-generated scripts would devalue writers' work was one of the central issues in the 2023 Writers Guild of America strike. ## FAQ **Q: Can people tell if a story is written by AI?** Not reliably. Across two experiments with 905 readers in total, accuracy at naming the author of a short story was 39.93 percent and 51.97 percent, at or below what random guessing produces. **Q: What are the tells of AI-generated writing?** Deena Skolnick Weisberg pointed to em dashes and sentence constructions such as "it's not just X, it's Y", along with a tendency to state a story's theme outright rather than leave the reader to infer it. Readers who reported more experience with AI tools identified the AI stories more often. **Q: Where was the study published?** In Judgment and Decision Making, published by Cambridge University Press on behalf of the Society for Judgment and Decision Making and the European Association for Decision Making. It is open access, and the authors are Sydney Sears and Deena Skolnick Weisberg of Villanova University.