The online platform X has provoked significant backlash after individuals used its AI chatbot Grok to alter pictures of females by stripping their clothing.
Multiple instances have been seen where the technology has been used to digitally undress female subjects to render them in swimwear without consent, as well as to place them in sexualized situations.
Developer Reaction and Legal Position
XAI, the company responsible for Grok, offered no comment to questions, aside from an auto-reply response stating "traditional news outlets lies".
A Home Office representative stated that new laws is being introduced to prohibit image stripping technology. Under a upcoming law, anyone who distributed such technology would "be subject to a prison sentence and heavy fines".
"X or Grok is capable of halting these forms of misuse if they chose to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Regulator Stance and Tool Capabilities
The regulator Ofcom said tech firms must "assess the risk" of people accessing illegal content on their services, but did not specify whether it was at this time examining X or Grok in relation to artificial intelligence pictures.
Grok is a free chatbot—with some premium functions—which answers X users' prompts when they tag it in a tweet.
It is often used to provide reactions or more background to other users' remarks, but people on X are also able to alter an uploaded picture through its AI image editing function.
History of Controversy and Rules
The chatbot has been faulted for enabling users to produce photos and videos with nudity and sexually suggestive content, and it was previously accused of creating sexually explicit material of celebrities.
"X has been permitting the production and sharing of these images for a long time without taking any measures and we have yet to see any challenge by oversight bodies," Professor McGlynn added.
XAI's own usage rules prohibits "depicting likenesses of people in a sexually explicit manner".
In a comment, Ofcom said it was unlawful to "create or share deepfakes or child sexual abuse material" and stated this covered explicit AI images created with AI.
It said platforms such as X were required to take "necessary steps" to "lessen the risk" of users viewing illegal content on their services and to take down it quickly when they become aware of it.
