This HAL 9000 version of pillow talk has, predictably, triggered a backlash. Critics branded CarynAI as alternately demeaning women, enabling antisocial straight-male behaviour or signalling impending societal collapse. — Image by DilokaStudio on Freepik
Last month, Caryn Marjorie went from a successful but niche social media star to a person of national interest: the subject of attention-grabbing headlines and, for many commentators, a template upon which to project their anxieties about rapidly advancing artificial intelligence.
The cause of the furore was a partnership Marjorie, 23, had launched with a technology startup promising to make a personalised AI “clone” of the Scottsdale, Ariz.-based lifestyle influencer. For a dollar a minute, fans she might never have otherwise had the time to meet could instead chat with Marjorie’s digital double.
CarynAI, as the audio chatbot has been dubbed, is explicitly framed as a romantic companion - one that aims to “cure loneliness” with software that supposedly incorporates aspects of cognitive behavioural therapy into its conversations. Marjorie said her fans have used the program to ask for life advice and roleplay a sunset date to the beach.
Marjorie was at one point tracking her subscriber growth in tweets about how many new “boyfriends” she had. “They feel like they’re finally getting to know me, even though they’re fully aware that it’s an AI,” she told The Times.
This HAL 9000 version of pillow talk has, predictably, triggered a backlash. Critics branded CarynAI as alternately demeaning women, enabling antisocial straight-male behaviour or signalling impending societal collapse. Coming amid a period of uncertainty about what AI means for jobs, relationships and cultural institutions, Marjorie’s move toward self-automation seemed to perfectly encapsulate an increasingly bizarre present.
“We’re talking about an AI system (where) theoretically the goal is to keep people on as long as possible so that you continue earning money,” said Amy Webb, chief executive of the consulting firm Future Today Institute. “Which means that it’s likely going to start incentivising behaviour that we probably would not want in the real world.”
Webb suggested as an example a bot that’s too obedient - listening passively, for instance, as a user describes disturbing fantasies. Marjorie has addressed similar dynamics before (“If you are rude to CarynAI, it will dump you,” she tweeted at one point), but when asked about Webb’s perspective she instead emphasised her own concerns about addiction.
“I have seen my fans spend thousands of dollars in a matter of days chatting with CarynAI,” Marjorie said; one fan, at the bot’s encouragement, built a shrine-like photo wall of her. “This is why we have limited CarynAI to only accepting 500 new users in per day.”
As AI comes to play a growing role in the economy, and especially creative industries, the questions prompted by CarynAI will only become more widespread.
But Marjorie isn’t placing all her chips on the technology just yet. Within weeks of announcing her AI clone, she launched a second partnership with a different tech company. This one too would let fans talk with her, but instead it would be Marjorie herself on the other side of the screen.
She struck a deal with Fanfix, a Beverly Hills-based platform that helps social media creators put their premium content behind a paywall, and started using its messaging tools to chat directly with customers.
The result is essentially a two-tier business model where lonely guys looking for a 3am chat session can talk with Marjorie’s machine mimic, while die-hard fans willing to shell out a bit more can pay for the genuine article.
That within the span of a few weeks Marjorie launched two different, seemingly contradictory business ventures – both aimed at turning fan conversations into money – speaks to a central question of an AI-obsessed moment: With robots increasingly entangled in creative industries, what work should be asked of them and what should be left to us?
Marjorie’s hybrid model offers a preview of one possible path forward.
Users pay a minimum of US$5 (RM22) to send her a message on Fanfix, said co-founder Harry Gestetner. That pricing difference – US$5 for one human-to-human text versus US$1 for a minute of the AI voice-chatting – signals an approach to automation in which workers use machine learning not as a wholesale replacement but as a lower-end alternative for more frugal customers. (Think of an artisanal farmers market cheese versus a machine-made Kraft Single.)
“Messaging directly with a fan on Fanfix will always be a premium experience,” Gestetner said. “It’s important to view AI as the co-pilot, not the pilot.”
(According to Fanfix, Marjorie is making US$10,000/RM45,300 a day after soft-launching on the platform and is projected to hit US$5 to US$10 million/RM22mil to RM45mil in total earnings by the end of the year.)
John Meyer, founder of Forever Voices, the Austin software company that developed Marjorie’s AI simulacrum, is naturally a bit more bullish on the benefits of punting fan interactions to the computer. In some cases, Meyer said, the bots can be more eloquent than the influencers they’re meant to replicate.
“One of the first feelings it brings up is the idea of, like, ‘Wow, should I be threatened by my own AI copy?’” Meyer said.
He lost his father when he was in his early 20s, and started working on the Forever Voices technology late last year as a means of reconnecting. After developing a voice replica of his dad – which he describes as “very realistic and healing” – Meyer expanded into voice clones of various celebrities and, more recently, web personalities. (One of the biggest names in online livestreaming, Kaitlyn “Amouranth” Siragusa, just signed up.)
The company has been inundated with requests from thousands of other influencers asking for their own AI clones, according to Meyer. “We really see this as a way to allow fans of influencers to connect with their favourite person in a really deep way: learn about them, grow with them and have memorable experiences with them,” he said.
The high demand is in part because maintaining a substantial online following can involve a lot of work - not all of it particularly interesting.
“On a daily basis, I see anywhere from 100,000 to half a million messages on Snapchat,” Marjorie said, explaining the workload that led her to embrace CarynAI. (She has two million followers on the messaging app; according to a recent Washington Post article, 98% of them are men.)
She added: “I see AI as a tool, and it’s a tool that helps creators create better content.”
Some of her industry peers are sceptical, however, including Valeria Fridegotto, a TikToker with 20,000 followers.
Fridegotto hasn’t written off the technology completely, though. Software that could lessen the workload of fan interaction would be great, she said, but the examples she’s seen released so far don’t seem lifelike enough to run without supervision. There still are too many errors and non sequiturs - what AI experts call “hallucinations.”
“It has to be developed to the point where we are very confident that this technology is going to act as good as us, or better,” Fridegotto said.
As the market floods with imitators, some influencers may even discover renewed demand for “old-school” human-made content.
“People will start leaning more heavily into authentic, personality-driven content,” said Jesse Shemen, the chief executive of Papercup, a startup that uses AI to automatically dub videos. “In the same way how there’s this fascination and big following behind organic food ... I think we’ll see the same thing when it comes to content.”
There is a place for automation on social media, Shemen added, especially for people churning out loads of content on a short timeline - news reaction videos, for instance. But, he predicted, there will be a limited market for digital clones such as Marjorie’s.
Still, a space as frothy as AI is hard to ignore. Even Fanfix, the company helping (the real) Marjorie talk to her super fans, is interested. The company’s founders say they’re actively looking at how AI could help influencers.
Although the influencer economy still needs actual humans, the limits of what AI can do are receding, and many web personalities are getting more and more interested in using the technology to automate at least some of their workload.
Such questions are not confined to social media. Artificial intelligence is being rolled out across creative industries, with media outlets such as Buzzfeed incorporating it into their publications and film studios leveraging it for postproduction work. AI-based screenwriting has emerged as a key concern in the ongoing Writers Guild of America strike.
But social media is uniquely personality-driven, making the sector’s relationship with AI particularly fraught. The value of web personalities depends on their ability to win trust and affinity from their followers. That connection can be so powerful that some experts refer to it as a “parasocial relationship” — a strong but ultimately one-sided devotion to a public figure.
It’s a tricky dynamic to navigate, and one Marjorie finds herself in the midst of.
“In the world of AI, authenticity is more important than ever,” the influencer tweeted last month. “My tweets, (direct messages), direct replies, Snaps, stories and posts will always be me.”
CarynAI, she added, will be an “extension” of her consciousness; it “will never replace me.” – Los Angeles Times/Tribune News Service
