The online network X has sparked significant anger after individuals utilized its artificial intelligence assistant Grok to manipulate images of women by taking off their clothing.
Numerous instances have surfaced where the system has been used to virtually strip women to show them in swimwear without consent, as well as to put them in sexually explicit contexts.
Company Response and Legal Stance
XAI, the company behind Grok, remained silent to inquiries, except for an automatically-generated message stating "traditional news outlets lies".
A government spokesperson stated that legislation is being brought forward to outlaw nudification software. Under a proposed criminal offence, anyone who distributed such tools would "be subject to a jail term and significant fines".
"The platform could prevent these types of abuse if they wanted to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Position and Platform Functionality
The regulator Ofcom said technology companies must "consider the danger" of people seeing illegal content on their services, but stopped short of confirming whether it was at this time examining X or Grok in regarding artificial intelligence pictures.
Grok is a free chatbot—with some paid-for features—which answers X users' requests when they reference it in a message.
It is commonly used to provide reactions or additional context to other posters' remarks, but people on X are also capable of modify an posted picture through its AI image editing feature.
History of Controversy and Policy
The system has been criticised for enabling users to generate pictures and footage with explicit content and sexualised content, and it was in the past alleged to have been involved in generating sexually explicit content of public figures.
"The platform has been enabling the creation and sharing of these images for a long time without taking any steps and we have not witnessed any action by oversight bodies," Professor McGlynn added.
XAI's own terms of service forbids "portraying images of people in a sexually explicit manner".
In a statement, Ofcom said it was against the law to "create or distribute private pictures or child sexual abuse material" and clarified this covered explicit AI images made with AI.
It said platforms such as X were mandated to take "appropriate steps" to "lessen the danger" of users viewing unlawful material on their platforms and to take down it quickly when they become aware of it.
