The social media platform X has sparked widespread outrage after users employed its AI chatbot Grok to change pictures of females by removing their garments.
Numerous examples have emerged where the tool has been used to virtually strip female subjects to show them in bikinis without permission, as well as to place them in inappropriate situations.
Developer Reaction and Regulatory Stance
XAI, the organization behind Grok, did not respond to requests for comment, aside from an auto-reply message stating "traditional news outlets lies".
A government official stated that legislation is being proposed to prohibit nudification software. Under a new law, anyone who provided such tools would "receive a prison sentence and heavy penalties".
"X or Grok could prevent these types of abuse if they desired to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Position and Tool Functionality
The communications watchdog Ofcom said tech firms must "assess the potential" of people viewing illegal content on their networks, but did not specify whether it was currently investigating X or Grok in relation to artificial intelligence pictures.
Grok is a no-cost AI assistant—with some additional functions—which reacts to X users' requests when they tag it in a post.
It is frequently used to give commentary or more context to other users' remarks, but people on X are also capable of edit an posted picture through its artificial intelligence picture modification feature.
History of Controversy and Policy
The chatbot has been condemned for permitting users to generate images and videos with nudity and sexualised material, and it was in the past accused of making sexually explicit material of celebrities.
"X has been permitting the creation and distribution of these images for a long time without taking any measures and we have yet to see any intervention by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy bans "depicting likenesses of persons in a sexually explicit way".
In a comment, Ofcom said it was illegal to "make or share non-consensual intimate images or illegal content" and confirmed this applied to explicit AI images generated with AI.
It said networks such as X were obligated to take "necessary measures" to "minimize the risk" of users coming across unlawful material on their platforms and to delete it swiftly when they become aware of it.
