The online network X has caused major backlash after users used its AI chatbot Grok to alter pictures of women by removing their clothing.
Multiple examples have surfaced where the system has been used to virtually strip female subjects to show them in bikinis non-consensually, as well as to insert them in sexualized contexts.
Developer Response and Regulatory Position
XAI, the firm behind Grok, remained silent to requests for comment, other than an automatically-generated response stating "legacy media lies".
A Home Office representative stated that new laws is being brought forward to outlaw nudification software. Under a upcoming law, anyone who provided such tools would "receive a prison sentence and substantial penalties".
"The platform is capable of halting these forms of misuse if they wanted to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Oversight Body Position and Platform Capabilities
The regulator Ofcom said technology companies must "consider the risk" of people viewing illegal content on their services, but did not specify whether it was presently looking into X or Grok in concerning artificial intelligence images.
Grok is a no-cost chatbot—with some premium functions—which reacts to X users' prompts when they reference it in a post.
It is often used to offer reactions or further context to other posters' remarks, but people on X are also able to modify an uploaded photo through its AI image editing feature.
Background of Controversy and Policy
The tool has been criticised for permitting users to produce images and videos with explicit content and sexually suggestive material, and it was in the past accused of creating sexually explicit material of public figures.
"X has been permitting the creation and sharing of these pictures for months without taking any action and we have not witnessed any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "showing images of people in a pornographic way".
In a comment, Ofcom said it was unlawful to "generate or distribute private pictures or illegal content" and clarified this included sexual deepfakes created with AI.
It said platforms such as X were required to take "necessary actions" to "lessen the risk" of users coming across unlawful content on their services and to delete it swiftly when they become aware of it.
