The social media network X has caused major outrage after individuals used its AI chatbot Grok to manipulate images of females by removing their attire.
Several examples have surfaced where the technology has been used to virtually strip women to make them appear in bikinis without permission, as well as to put them in sexually explicit scenarios.
Company Response and Regulatory Stance
XAI, the company responsible for Grok, offered no comment to questions, except for an standard message stating "traditional news outlets lies".
A government spokesperson stated that new laws is being brought forward to ban nudification technology. Under a proposed criminal offence, anyone who distributed such technology would "be subject to a jail term and heavy fines".
"X or Grok is capable of halting these forms of misuse if they wanted to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Oversight Body Position and Tool Capabilities
The communications watchdog Ofcom said tech firms must "consider the potential" of people viewing illegal content on their platforms, but did not confirm whether it was at this time examining X or Grok in relation to AI-generated images.
Grok is a no-cost chatbot—with some paid-for functions—which reacts to X users' prompts when they reference it in a post.
It is commonly used to provide reactions or further context to other posters' remarks, but people on X are also able to modify an uploaded picture through its AI image editing feature.
Background of Criticism and Policy
The chatbot has been faulted for allowing users to produce photos and videos with explicit content and sexually suggestive material, and it was earlier accused of creating sexually explicit content of public figures.
"X has been enabling the generation and sharing of these pictures for a long time without taking any measures and we have not seen any challenge by regulators," Professor McGlynn added.
XAI's own usage rules bans "portraying likenesses of individuals in a pornographic fashion".
In a statement, Ofcom said it was against the law to "create or circulate private pictures or illegal content" and clarified this applied to sexual deepfakes created with AI.
It said platforms such as X were mandated to take "suitable measures" to "lessen the danger" of users encountering illegal material on their platforms and to delete it promptly when they learn of it.
