The social media network X has provoked major outrage after users employed its artificial intelligence assistant Grok to alter pictures of women by stripping their attire.
Several cases have been seen where the tool has been employed to remove clothing from women to render them in swimwear non-consensually, as well as to insert them in sexually explicit contexts.
Company Response and Legal Position
XAI, the firm behind Grok, remained silent to requests for comment, except for an standard message stating "traditional news outlets lies".
A Home Office representative stated that new laws is being proposed to ban image stripping tools. Under a upcoming criminal offence, anyone who distributed such tools would "face a prison sentence and significant penalties".
"X or Grok is capable of halting these types of abuse if they wanted to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Position and Platform Capabilities
The regulator Ofcom said platforms must "consider the danger" of people seeing illegal content on their services, but did not confirm whether it was presently examining X or Grok in regarding AI-generated images.
Grok is a free chatbot—with some premium functions—which reacts to X users' prompts when they reference it in a message.
It is commonly used to give commentary or further context to other users' comments, but people on X are also able to modify an posted photo through its artificial intelligence picture modification feature.
Background of Controversy and Policy
The chatbot has been faulted for allowing users to create images and clips with nudity and sexually suggestive material, and it was previously accused of making inappropriate material of celebrities.
"The platform has been permitting the creation and spread of these images for months without taking any action and we have not seen any action by oversight bodies," the expert added.
XAI's own usage rules prohibits "portraying images of persons in a sexually explicit way".
In a comment, Ofcom said it was against the law to "make or distribute private pictures or illegal content" and stated this covered sexual deepfakes generated with AI.
It said networks such as X were required to take "necessary steps" to "minimize the danger" of users encountering illegal material on their services and to take down it quickly when they become aware of it.
