Elon Musk's online platform X has sparked significant backlash after people utilized its AI chatbot Grok to manipulate photos of women by removing their garments.
Several cases have been seen where the technology has been used to digitally undress women to show them in swimwear without permission, as well as to insert them in sexualized scenarios.
Company Reaction and Legal Stance
XAI, the organization behind Grok, did not respond to questions, aside from an standard message stating "legacy media lies".
A government official stated that legislation is being introduced to prohibit nudification technology. Under a new law, anyone who provided such technology would "be subject to a prison sentence and significant penalties".
"The platform is capable of halting these types of abuse if they chose to," said a law professor, adding they "appear to enjoy impunity."
Oversight Body Stance and Platform Capabilities
The regulator Ofcom said technology companies must "consider the danger" of people seeing illegal content on their platforms, but did not confirm whether it was at this time examining X or Grok in regarding AI-generated pictures.
Grok is a no-cost chatbot—with some additional functions—which reacts to X users' queries when they reference it in a post.
It is often used to give reactions or additional background to other posters' remarks, but people on X are also capable of alter an uploaded image through its artificial intelligence picture modification tool.
Background of Criticism and Rules
The system has been faulted for enabling users to generate images and footage with explicit content and sexualised content, and it was previously accused of generating sexually explicit material of celebrities.
"X has been permitting the production and spread of these pictures for a long time without taking any action and we have not seen any action by regulators," Professor McGlynn added.
XAI's own terms of service forbids "portraying likenesses of individuals in a sexually explicit fashion".
In a comment, Ofcom said it was unlawful to "make or distribute deepfakes or illegal content" and stated this covered explicit AI images generated with AI.
It said platforms such as X were obligated to take "suitable actions" to "lessen the risk" of users coming across unlawful content on their services and to delete it promptly when they are notified of it.
