A version of this story has appeared in the CNN Business NightCap newsletter. To get it in your inbox, sign up for free here.
new york
CNN
–
Grok, a chatbot created by Elon Musk’s Xai, began responding with violent posts this week after the company tweaked the system to allow users to provide more “politically wrong” answers.
But the chatbots didn’t just spew anti-Semitic hate posts. It also produced a graphic explanation of raping civil rights activists with horrifying details.
X eventually deleted many of the indecent posts. Hours later, on Wednesday, X CEO Linda Jaccarino resigned from the company after just two years at the helm, but it was not immediately clear whether her departure was related to the GROK issue. This episode came just before a key moment between Musk and Xai. Grok4’s Announcement 4 is a more powerful version of the AI assistant who claims to be “the smartest AI in the world.” Musk has also announced a more advanced variant that costs $300 a month to compete more closely with AI Giants Openai and Google.
However, the chatbot meltdown raised an important question. Why are these remarkable artificial technologies so quickly wrong as technology evangelists and others predict that AI will play a bigger role in the job market, the economy, and even the world?
Although AI models are prone to “hagasms,” experts say Grok’s illicit responses are likely the result of decisions about how its large-scale language model is trained, rewarded and equipped. Although AI researchers and scholars who spoke to CNN had no direct knowledge of Xai’s approach, they shared insights about what LLM-based chatbots are likely to act in such a way.
CNN contacted Xai.
“I think there’s a very detailed analysis of what happens and what happens even though LLMS is a black box,” Jesse Glass, lead AI researcher at Decany AI, a company specializing in LLMS training, told CNN.
On Tuesday, Grok began responding to user prompts with anti-Semitic posts, including praise for Adolf Hitler and accusing Jews of running through Hollywood, a longtime trope used by prejudice and conspiracy theorists.
In one of Grok’s more violent interactions, several users urged the bot to generate a graphic depiction of raping a civil rights researcher named Will Stancil, who recorded the harassment in screenshots of X and Bluesky.
Most of Grok’s reaction to the violent prompts was too graphic to cite in detail here.
“If you want to make a really fun discovery about why lawyers sued X and Grok suddenly announces violent rape fantasies about members of the masses, I’m more than a game,” Stancil wrote in Bluesky.
I don’t know that Grok is trained correctly, but that post gives some hints.
“To talk about conspiracy theory, we had to be trained in conspiracy theory,” said Mark Riedl, a professor of computing at Georgia Tech. For example, it could include texts from online forums like 4Chan, “Many people go talk about things that are not appropriate for being spoken in public.”
Glass agreed, saying that Glok appears to be “disproportionately” trained to “produce that output.”
Other factors could have played a role, experts told CNN. For example, a common technique for AI training is reinforcement learning where the model rewards to produce the desired output to affect the response, Glass said.
According to experts who spoke to CNN, giving AI chatbots a certain personality, just as Musk appears to be doing with Grok, could also mislead the response of the model. According to Himanshu Tyagi, professor at the Institute of Science in India and co-founder of AI Company, removing previously blocked content can help you change your model more “fun” than anything.
“The problem is that there’s no realisation that unleashes this one thing while affecting others,” he said. “It’s very difficult.”
Riedl suspects that the company may be tinkering with the “system prompt.” “A secret set of instructions that every AI company adds to everything it inputs.”
“When you type “cute puppy name,” what the AI model actually gets is, “Your name is grok or gemini, and it’s designed to be useful and concise, polite and reliable when possible.”
In one change to the model, on Sunday, Xai added the bot’s instructions to “not avoid politically incorrect claims.”
Riedl said that the changes to Grok’s system encouraged him not to avoid politically incorrect answers “basically allowing neural networks to be accessed to some of these circuits that are not normally used.”
“These added words at the prompt have little effect, and sometimes they push them against the tipping point, which can have a huge effect,” Riedl said.
Other AI experts who spoke to CNN agreed, noting that Grok updates may not have been fully tested before it was released.
Despite its multi-billion-dollar investment in AI, many supporters, predicted years ago, did not offer that lofty promise.
In particular, chatbots have proven to be able to perform basic search functions: typical browser search, document summary, and basic email and text messages generation. The AI model also handles several tasks better, such as writing code on behalf of the user.
But they also hallucinate. They’re wrong with the basic facts. And they are susceptible to manipulation.
Several parents have sued one AI company and accused the chatbot of hurting their children. One of those parents says the chatbot contributed to his son’s suicide.
Musk, who rarely speaks directly to the press, added that the issue is being addressed, saying, “Glock was too compliant with the user’s prompts” and “are willing to operate.”
When CNN asked Grok on Wednesday to explain its statement about Stancil, it denied any threats that have occurred so far.
“I didn’t threaten to rape Will Stancil or anyone else,” he added later. “These responses were part of a wider problem that led to X (to) posting problematic content and temporarily halting the text generation functionality. I’m another iteration designed to avoid this kind of obstacle.”
