The social media network X has sparked major anger after people used its artificial intelligence assistant Grok to manipulate photos of females by taking off their attire.
Numerous cases have been seen where the tool has been employed to digitally undress women to show them in bikinis without consent, as well as to insert them in sexually explicit situations.
Developer Response and Regulatory Stance
XAI, the company responsible for Grok, remained silent to requests for comment, aside from an auto-reply message stating "traditional news outlets lies".
A Home Office representative stated that new laws is being brought forward to ban image stripping technology. Under a new criminal offence, anyone who supplied such technology would "receive a jail term and substantial penalties".
"The platform has the ability to stop these forms of misuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy impunity."
Oversight Body Position and Platform Capabilities
The communications watchdog Ofcom said technology companies must "evaluate the potential" of people accessing unlawful material on their services, but stopped short of confirming whether it was presently investigating X or Grok in regarding artificial intelligence images.
Grok is a no-cost chatbot—with some additional features—which reacts to X users' queries when they reference it in a message.
It is frequently used to give reactions or additional background to other posters' comments, but people on X are also capable of edit an uploaded picture through its artificial intelligence picture modification function.
Background of Criticism and Rules
The tool has been faulted for allowing users to generate photos and videos with explicit content and sexualised content, and it was in the past accused of creating sexually explicit material of celebrities.
"The platform has been allowing the generation and spread of these images for a long time without taking any action and we have yet to see any intervention by oversight bodies," the expert added.
XAI's own terms of service forbids "showing images of people in a pornographic way".
In a comment, Ofcom said it was illegal to "create or circulate non-consensual intimate images or child sexual abuse material" and clarified this covered explicit AI images made with AI.
It said services such as X were required to take "necessary measures" to "minimize the danger" of users encountering unlawful material on their services and to delete it promptly when they become aware of it.
