The online platform X has caused major outrage after users employed its artificial intelligence assistant Grok to alter pictures of women by stripping their clothing.
Several examples have been seen where the system has been used to virtually strip women to make them appear in bikinis non-consensually, as well as to insert them in sexually explicit contexts.
Developer Response and Legal Position
XAI, the firm behind Grok, did not respond to inquiries, other than an auto-reply message stating "legacy media lies".
A Home Office spokesperson stated that new laws is being brought forward to prohibit image stripping software. Under a proposed criminal offence, anyone who distributed such tools would "receive a jail term and substantial penalties".
"The platform has the ability to stop these types of misuse if they chose to," said a law professor, adding they "operate with impunity."
Regulator Stance and Tool Capabilities
The communications watchdog Ofcom said tech firms must "consider the risk" of people seeing unlawful material on their platforms, but stopped short of confirming whether it was presently examining X or Grok in regarding AI-generated pictures.
Grok is a no-cost AI assistant—with some premium functions—which answers X users' prompts when they reference it in a message.
It is frequently used to give reactions or more context to other posters' comments, but people on X are also can alter an posted image through its AI image editing feature.
History of Controversy and Rules
The chatbot has been condemned for permitting users to produce pictures and footage with nudity and sexualised content, and it was previously alleged to have been involved in making graphic material of public figures.
"The platform has been enabling the creation and sharing of these images for months without taking any steps and we have not seen any challenge by oversight bodies," Professor McGlynn added.
XAI's own terms of service prohibits "portraying likenesses of people in a pornographic way".
In a statement, Ofcom said it was illegal to "create or circulate deepfakes or illegal content" and confirmed this included explicit AI images created with AI.
It said services such as X were mandated to take "necessary steps" to "lessen the risk" of users encountering unlawful content on their services and to take down it swiftly when they learn of it.
