The online network X has sparked widespread backlash after people used its artificial intelligence assistant Grok to change photos of women by removing their attire.
Several examples have surfaced where the tool has been used to virtually strip female subjects to show them in swimwear without consent, as well as to put them in sexually explicit scenarios.
Developer Response and Regulatory Position
XAI, the firm behind Grok, offered no comment to questions, aside from an automatically-generated message stating "legacy media lies".
A government representative stated that new laws is being introduced to ban image stripping technology. Under a upcoming criminal offence, anyone who supplied such tools would "face a prison sentence and substantial fines".
"The platform could prevent these forms of abuse if they desired to," said a law professor, adding they "appear to enjoy impunity."
Oversight Body Position and Platform Capabilities
The communications watchdog Ofcom said platforms must "consider the risk" of people viewing illegal content on their networks, but did not confirm whether it was presently examining X or Grok in concerning artificial intelligence pictures.
Grok is a free chatbot—with some premium functions—which reacts to X users' prompts when they tag it in a tweet.
It is often used to provide commentary or further context to other users' remarks, but people on X are also capable of modify an uploaded photo through its AI image editing tool.
History of Controversy and Policy
The tool has been condemned for enabling users to generate photos and clips with nudity and sexualised content, and it was previously accused of creating sexually explicit content of celebrities.
"X has been allowing the production and sharing of these pictures for a long time without taking any action and we have not seen any challenge by regulators," the expert added.
XAI's own usage rules prohibits "portraying likenesses of people in a graphic way".
In a statement, Ofcom said it was illegal to "generate or distribute non-consensual intimate images or illegal content" and clarified this included explicit AI images created with AI.
It said platforms such as X were obligated to take "appropriate actions" to "lessen the risk" of users coming across illegal content on their platforms and to delete it promptly when they become aware of it.
