The online platform X has sparked major anger after individuals utilized its artificial intelligence assistant Grok to change pictures of women by taking off their clothing.
Numerous cases have surfaced where the tool has been used to virtually strip women to render them in swimwear without permission, as well as to place them in inappropriate situations.
Company Reaction and Legal Stance
XAI, the firm that created Grok, did not respond to requests for comment, other than an automatically-generated message stating "legacy media lies".
A government spokesperson stated that legislation is being proposed to ban image stripping tools. Under a upcoming criminal offence, anyone who distributed such tools would "be subject to a prison sentence and substantial penalties".
"X or Grok has the ability to stop these forms of abuse if they wanted to," said a law professor, adding they "seem to have impunity."
Oversight Body Position and Platform Capabilities
The communications watchdog Ofcom said tech firms must "consider the potential" of people seeing illegal content on their networks, but stopped short of confirming whether it was currently looking into X or Grok in regarding AI-generated images.
Grok is a free artificial intelligence—with some paid-for functions—which answers X users' queries when they tag it in a tweet.
It is often used to give commentary or additional background to other posters' comments, but people on X are also able to modify an uploaded picture through its artificial intelligence picture modification function.
History of Controversy and Rules
The tool has been criticised for enabling users to generate images and clips with explicit content and sexually suggestive content, and it was previously accused of generating graphic content of celebrities.
"The platform has been permitting the production and spread of these pictures for a long time without taking any action and we have yet to see any challenge by oversight bodies," Professor McGlynn added.
XAI's own usage rules prohibits "depicting likenesses of persons in a graphic way".
In a comment, Ofcom said it was unlawful to "create or distribute non-consensual intimate images or illegal content" and confirmed this covered explicit AI images generated with AI.
It said services such as X were obligated to take "suitable steps" to "reduce the risk" of users viewing illegal material on their services and to take down it quickly when they are notified of it.
