The online network X has caused significant outrage after users utilized its AI chatbot Grok to manipulate photos of women by taking off their clothing.
Numerous examples have been seen where the tool has been employed to digitally undress women to make them appear in bikinis non-consensually, as well as to place them in sexually explicit situations.
Company Reaction and Legal Position
XAI, the firm behind Grok, did not respond to questions, aside from an auto-reply response stating "legacy media lies".
A government official stated that new laws is being brought forward to ban image stripping technology. Under a upcoming law, anyone who distributed such technology would "be subject to a prison sentence and substantial fines".
"X or Grok could prevent these types of abuse if they chose to," said a law professor, adding they "seem to have a lack of consequences."
Oversight Body Stance and Platform Capabilities
The regulator Ofcom said platforms must "evaluate the danger" of people viewing illegal content on their services, but did not confirm whether it was presently looking into X or Grok in concerning AI-generated pictures.
Grok is a free AI assistant—with some paid-for functions—which responds to X users' prompts when they reference it in a tweet.
It is often used to offer commentary or additional context to other posters' remarks, but people on X are also can modify an uploaded photo through its AI image editing tool.
History of Controversy and Policy
The system has been condemned for permitting users to generate photos and footage with nudity and sexualised content, and it was previously alleged to have been involved in generating sexually explicit content of celebrities.
"X has been enabling the generation and distribution of these pictures for a long time without taking any measures and we have not witnessed any intervention by oversight bodies," the expert added.
XAI's own acceptable use policy forbids "portraying images of individuals in a sexually explicit fashion".
In a statement, Ofcom said it was unlawful to "create or share deepfakes or child sexual abuse material" and clarified this covered explicit AI images generated with AI.
It said networks such as X were mandated to take "suitable steps" to "minimize the danger" of users coming across unlawful material on their services and to remove it quickly when they become aware of it.
