AI- or Human-Written Scripts? It May Not Make a Difference to Viewers

The Hollywood writers and actors strike has dominated the news this summer. The use — or misuse — of generative artificial intelligence lies at the heart of the matter. SAG-AFTRA and the Writers Guild of America are demanding that their contracts include AI regulations to protect writers, their creativity and their original works.

Amidst this conversation, University of Florida College of Journalism and Communications Advertising Associate Professor and Interim Director of the STEM Translational Communication Center Benjamin K. Johnson and CJC graduate students Rachel Son, Qingyuan Yang, and Benjamin T. Vollmer wondered if audiences gained as much enjoyment from machine-generated narratives as they do from narratives created by humans.

The researchers set out to gauge audience reactions to works of short fiction written by ChatGPT and a version of the ChatGPT story edited by the researchers themselves. ChatGPT created a short narrative based on a prompt to write a 1,000-word short story. The human authors used the same storyline but warmed the tone by embellishing the story with literary devices — metaphor, simile and vivid imagery. Both stories were told in the third person, and the audience spent about three minutes reading the story.

They looked at two specific things: how engaged the audience was with the human-authored and bot-authored stories and how the label of human-authored and bot-authored affected that engagement.

Study participants were randomly assigned human-authored, human-labeled, bot-authored and bot-labeled stories. Their responses were based on narrative engagement, enjoyment, perceived authenticity, perceived humanness and mind perception.

Surprisingly, the readers enjoyed the stories written by bots as much as they did the stories written by humans.

Even more notable was that the audience enjoyed the stories more if they thought they were written by humans. Their perception of a machine author colored their experience with the text, even when the content was identical to a human author. Labels that disclosed authorship were more important than content.

As dozens of sci-fi movies (and valid research) have shown us, humans are more accepting of artificial intelligence when it possesses anthropomorphic qualities. The authors of this study write, “The outcomes of our research could suggest that a physical aspect, such as a robot’s humanoid appearance or voice, might be necessary to impact the level of narrative engagement as opposed to just a label.”

The original paper, “Engaging with Bot Narratives: The Audience Experience and Perceptions of Machine Script Writers,” was delivered at the 73rd Annual International Communication Association Conference, Toronto, Canada, on May 25, 2023.

Authors: Rachel Son, Qingyuan Yang, Benjamin T. Vollmer and Benjamin K. Johnson

This summary was written by Gigi Marino.

Posted: August 28, 2023
Tagged as: AIatUF, Benjamin Johnson, Benjamin Vollmer, Qingyuan Yang, Rachel Son