The online network X has caused widespread outrage after people utilized its AI chatbot Grok to alter photos of women by taking off their garments.
Multiple instances have surfaced where the system has been used to remove clothing from female subjects to make them appear in bikinis without consent, as well as to put them in sexualized scenarios.
Developer Response and Legal Stance
XAI, the organization behind Grok, offered no comment to requests for comment, aside from an automatically-generated response stating "legacy media lies".
A government representative stated that new laws is being brought forward to prohibit image stripping tools. Under a new criminal offence, anyone who distributed such technology would "be subject to a prison sentence and heavy fines".
"X or Grok has the ability to stop these forms of abuse if they chose to," said a law professor, adding they "seem to have impunity."
Oversight Body Position and Tool Functionality
The regulator Ofcom said platforms must "evaluate the danger" of people accessing illegal content on their networks, but did not confirm whether it was at this time looking into X or Grok in regarding AI-generated pictures.
Grok is a free AI assistant—with some paid-for features—which reacts to X users' prompts when they reference it in a message.
It is often used to provide commentary or further background to other users' comments, but people on X are also capable of modify an posted photo through its artificial intelligence picture modification feature.
History of Controversy and Policy
The system has been faulted for enabling users to generate photos and videos with nudity and sexualised content, and it was previously alleged to have been involved in making graphic content of celebrities.
"X has been enabling the creation and distribution of these pictures for months without taking any measures and we have not witnessed any challenge by oversight bodies," Professor McGlynn added.
XAI's own terms of service prohibits "depicting images of persons in a sexually explicit way".
In a comment, Ofcom said it was unlawful to "create or share deepfakes or illegal content" and confirmed this included sexual deepfakes made with AI.
It said platforms such as X were obligated to take "suitable actions" to "minimize the risk" of users encountering illegal content on their platforms and to remove it quickly when they are notified of it.
