The social media network X has provoked major backlash after people utilized its AI chatbot Grok to change photos of females by stripping their clothing.
Multiple cases have surfaced where the technology has been employed to remove clothing from female subjects to show them in swimwear without consent, as well as to put them in sexualized situations.
Developer Reaction and Legal Stance
XAI, the organization that created Grok, remained silent to inquiries, except for an standard response stating "legacy media lies".
A government representative stated that legislation is being brought forward to ban image stripping tools. Under a proposed criminal offence, anyone who distributed such tools would "be subject to a prison sentence and significant fines".
"X or Grok could prevent these types of misuse if they desired to," said a law professor, adding they "operate with a lack of consequences."
Regulator Position and Tool Capabilities
The regulator Ofcom said technology companies must "consider the potential" of people viewing unlawful material on their platforms, but did not specify whether it was presently examining X or Grok in concerning artificial intelligence images.
Grok is a free chatbot—with some additional features—which reacts to X users' prompts when they mention it in a tweet.
It is frequently used to provide reactions or more context to other posters' comments, but people on X are also can modify an posted picture through its artificial intelligence picture modification feature.
Background of Criticism and Policy
The chatbot has been condemned for permitting users to create photos and videos with nudity and sexually suggestive material, and it was in the past alleged to have been involved in creating sexually explicit material of celebrities.
"The platform has been allowing the creation and spread of these images for months without taking any action and we have not seen any challenge by oversight bodies," Professor McGlynn added.
XAI's own terms of service forbids "portraying images of individuals in a sexually explicit way".
In a statement, Ofcom said it was unlawful to "create or circulate private pictures or child sexual abuse material" and clarified this included sexual deepfakes created with AI.
It said services such as X were required to take "appropriate steps" to "lessen the risk" of users coming across illegal content on their services and to take down it promptly when they learn of it.
