The social media network X has sparked widespread outrage after individuals utilized its AI chatbot Grok to manipulate photos of women by taking off their clothing.
Several examples have surfaced where the technology has been employed to remove clothing from female subjects to show them in bikinis non-consensually, as well as to put them in sexually explicit contexts.
Developer Reaction and Legal Position
XAI, the organization behind Grok, did not respond to questions, aside from an auto-reply response stating "traditional news outlets lies".
A government representative stated that new laws is being proposed to outlaw image stripping technology. Under a upcoming criminal offence, anyone who distributed such technology would "be subject to a prison sentence and significant penalties".
"X or Grok has the ability to stop these types of abuse if they desired to," said a law professor, adding they "appear to enjoy a lack of consequences."
Oversight Body Position and Platform Capabilities
The communications watchdog Ofcom said technology companies must "consider the risk" of people viewing unlawful material on their networks, but stopped short of confirming whether it was presently examining X or Grok in relation to artificial intelligence images.
Grok is a free AI assistant—with some additional functions—which responds to X users' queries when they reference it in a message.
It is commonly used to give reactions or additional background to other users' comments, but people on X are also capable of edit an posted photo through its AI image editing tool.
History of Criticism and Rules
The chatbot has been faulted for allowing users to produce images and footage with nudity and sexually suggestive content, and it was earlier alleged to have been involved in generating inappropriate material of celebrities.
"The platform has been permitting the generation and sharing of these pictures for a long time without taking any action and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own acceptable use policy bans "showing images of people in a graphic way".
In a comment, Ofcom said it was unlawful to "generate or share private pictures or illegal content" and clarified this applied to explicit AI images created with AI.
It said platforms such as X were required to take "appropriate measures" to "lessen the danger" of users coming across unlawful content on their platforms and to delete it swiftly when they become aware of it.
