The viral images of Swift have helped push nonconsensual deepfakes to the top of the political agenda, after years of concerns from experts that the technology was getting out of hand. — AP
The pornographic deepfakes of Taylor Swift that proliferated on social media late last month originated from an online challenge to break safety mechanisms designed to block people from generating lewd images with artificial intelligence, according to social network analysis company Graphika.
For weeks, users of Internet forum 4chan have taken part in daily competitions to find words and phrases that could help them bypass the filters on popular image-generation services, which include Microsoft Designer and OpenAI’s Dall-E, the researchers found. The ultimate goal was to create sexual images of prominent female figures such as singers and politicians.
"While viral pornographic pictures of Taylor Swift have brought mainstream attention to the issue of AI-generated non-consensual intimate images, she is far from the only victim,” said Cristina López G., a senior analyst at Graphika, in an email. "In the 4chan community where these images originated, she isn’t even the most frequently targeted public figure. This shows that anyone can be targeted in this way, from global celebrities to school children.”
A representative for 4chan didn’t respond to a request for comment.
According to Graphika’s analysis, the deepfakes originated on that channel in early January before moving to Telegram and then X, which was forced to block all Taylor Swift searches after lewd pictures sparked outrage. One of the images remained on X for 17 hours and had more than 45 million views, according to the Verge.
Many text-to-image generation companies have banned certain keywords to stop users from generating harmful content. However, some users on 4chan and other online forums have shared tips on how to break through these safety measures.
"These images originated from a community of people motivated by the ‘challenge’ of circumventing the safeguards of generative AI products, and new restrictions are seen as just another obstacle to ‘defeat,’” López G. said. "It’s important to understand the gamified nature of this malicious activity in order to prevent further abuse at the source.”
The viral images of Swift have helped push nonconsensual deepfakes to the top of the political agenda, after years of concerns from experts that the technology was getting out of hand. The White House released a statement condemning the Swift images, and Congressman Joe Morelle, a Democrat from New York, said he hoped the incident would bring attention to his efforts to make deepfake pornography a federal crime that warrants fines and jail time.
On the 4chan channel, Swift is just one of many celebrities whose likeness was used to create sexual or violent images. Others include Billie Eilish, Ariana Grande and Emma Watson. Representatives for Swift, Eilish, Grande and Watson didn’t respond to requests for comment.
As of Jan 30, Swift deepfake requests were still being made, including at least one that sought violent imagery of her.
Celebrity deepfake pornography has been available on the Internet on numerous websites for years, but cases involving ordinary people are rising along with the sophistication and ease in which people can access the technology, according to experts. Often victims have little recourse and struggle to get the images taken down. – Bloomberg
