The online platform X has sparked major backlash after individuals used its AI chatbot Grok to alter photos of women by taking off their attire.
Numerous cases have been seen where the technology has been used to remove clothing from women to show them in swimwear without permission, as well as to put them in sexually explicit contexts.
Company Reaction and Legal Position
XAI, the company behind Grok, offered no comment to requests for comment, other than an automatically-generated response stating "legacy media lies".
A Home Office official stated that new laws is being brought forward to prohibit image stripping software. Under a new law, anyone who provided such tools would "be subject to a jail term and heavy penalties".
"The platform is capable of halting these forms of misuse if they chose to," said a law professor, adding they "seem to have a lack of consequences."
Regulator Stance and Platform Capabilities
The regulator Ofcom said tech firms must "evaluate the danger" of people seeing illegal content on their platforms, but stopped short of confirming whether it was at this time investigating X or Grok in regarding AI-generated images.
Grok is a free AI assistant—with some premium functions—which responds to X users' requests when they reference it in a tweet.
It is frequently used to give reactions or more context to other posters' remarks, but people on X are also capable of edit an posted photo through its AI image editing feature.
Background of Criticism and Rules
The tool has been criticised for allowing users to generate photos and videos with explicit content and sexualised material, and it was in the past accused of making sexually explicit material of public figures.
"X has been permitting the production and sharing of these pictures for months without taking any action and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "showing images of individuals in a sexually explicit way".
In a comment, Ofcom said it was against the law to "make or share deepfakes or child sexual abuse material" and clarified this covered sexual deepfakes created with AI.
It said networks such as X were mandated to take "appropriate measures" to "reduce the risk" of users viewing unlawful material on their services and to take down it promptly when they become aware of it.
