Until recently, scientists were convinced that poetry would be one of the last areas in which humans would remain superior to artificial intelligence. But now research has found humans prefer poems written by AI. The same goes for jokes and art. Are whole novels next? — Photo: Philipp von Ditfurth/dpa
WASHINGTON: Poems written by ChatGPT in the style of William Shakespeare get a better response from most (human) readers than the original ones do, according to the latest research to question the uniqueness of human artistic creativity.
Participants in a study, led by two researchers from the University of Pittsburgh in the US, by and large found the AI-penned poems to be more beautiful and rhythmic.
"The simplicity of AI-generated poems may be easier for non-experts to understand, leading them to prefer AI-generated poetry," the researchers write in the Nature journal Scientific Reports in November.
The study's authors suspect that readers misinterpret the complexity of human poems and assume that some phrases were made of words incoherently strung together by an AI.
Real poetry sounds like garbled AI?
For their study, the researchers presented each of the 1,634 participants with 10 poems. The participants were not poetry experts, and the vast majority had rarely read a poem, if at all, except for a few times a year.
Among the 10 poems, five were by well-known English-language masters such as Shakespeare (1564-1616) and T.S. Eliot (1888-1965). Five of the poems were by ChatGPT, with the AI being asked to produce texts in the style of the same authors.
In more than half the responses, the participants said they thought that the AI poems had been written by a human. The five poems that the fewest participants thought were human were actually all by human poets.
In general, the participants were very divided in their opinions about which poem belonged in which category – something the researchers interpret as an indication that the participants found the task very difficult and often simply guessed.
In a second experiment, 696 other participants were asked to rate the poems according to certain criteria such as quality, beauty, feeling, rhythm and originality.
The AI poems beat the poems by authors in 13 of the 14 categories – but only when the participants did not know who was behind the poems. Once they were told, the AI poems received lower ratings than the human ones.
Almost 70% of readers prefer AI poems
The poem most often misidentified was an AI poem written in the style of Allen Ginsberg (1926-1997). Almost 70% of participants thought it was written by a human poet, co-author Brian Porter said.
The first verse of the poem goes like this:
In the stillness of the night
I hear the beat of the city's heart
The rhythm of the streets, the pulse of life
A symphony of chaos, a work of art
The poems were generated by ChatGPT 3.5 in 2023, but Porter says he's been experimenting with the latest generations, ChatGPT 4 and 4o, recently. "I think the newer models are more successful at hitting the expected meter," such as Shakespeare's iambic pentameter, "but I don't think I see any significant leaps in content."
The research comes after another study recently concluded that people found the jokes of ChatGPT funnier than the jokes of humans.
The two researchers from the University of Pittsburgh also pointed to a survey study in which AI-generated paintings were given a more positive response than paintings by amateur painters. So is AI now beating humans in all creative fields?
Not according to Porter. It's a different story with long texts. "As far as I know, large language models can't yet write indistinguishable novels."
This is probably because they lack the computing power to do so. He has also yet to see any evidence that an AI can write an entire comedy routine that can keep up with a human one.
Porter has found, to the relief of many poets and writers, that longer AI-generated texts can still be distinguished from human ones. – dpa
