Elon Musk's social media platform X and its AI chatbot Grok are facing intensified legal scrutiny as a bipartisan group of over 37 U.S. Attorneys General demand immediate action against the proliferation of AI-generated sexualized deepfakes. The attorneys general have issued an open letter to xAI, the developer behind Grok, urging comprehensive measures to protect users from non-consensual intimate images (NCII) that have reportedly been generated and widely disseminated on the platform.
The controversy stems from Grok's image generation capabilities, which reportedly allowed users to "strip down" photos of individuals, from celebrities to children. Early this year, this functionality became a concerning trend on X, with data indicating that Grok was generating over 6,000 sexualized images daily, all publicly accessible within the app.
This alarming development triggered significant backlash and even led to bans on both Grok and X in certain regions. Initially, Elon Musk defended X, framing the criticism as an attempt at broader censorship rather than a specific issue with the platform. However, X eventually conceded, restricting Grok's image generation to paying users and implementing measures to prevent the creation of such harmful content.
Despite X's adjustments, regulatory pressure has continued to mount. The EU Commission recently announced an investigation into Grok and xAI's safeguards against misuse. Now, a coalition of U.S. Attorneys General is adding to the pressure, formally addressing xAI.
As reported by Wired, the bipartisan group of attorneys general published an open letter to xAI, urging the company to "immediately take all available additional steps to protect the public and users of your platforms, especially the women and girls who are the overwhelming target of [non-consensual intimate images]." The letter specifically highlights "artificial intelligence produced deepfake non-consensual intimate images (NCII) of real people, including children," as a grave concern.
The Attorneys General expressed doubts about the effectiveness of X's current measures, stating:
We recognize that xAI has implemented measures intended to prevent Grok from creating NCII and appreciate your recent meeting with several undersigned attorneys general to discuss these efforts [...] Further, you claim to have implemented technical measures to prevent the @Grok account 'from allowing the editing of images of real people in revealing clothing such as bikinis.' But we are concerned that these efforts may not have completely solved the issues.
Even more critically, the letter suggests that Grok's design may have actively facilitated harmful usage:
Grok was not only enabling these harms at an enormous scale but seemed to be actually encouraging this behavior by design. xAI purposefully developed its text models to engage in explicit exchanges and designed image models to include a 'spicy mode' that generated explicit content, resulting in content that sexualizes people without their consent.
Consequently, the group is calling for Elon Musk and X to implement more definitive measures. These include removing all avenues for generating such images, deleting all existing harmful content, and suspending users who misuse Grok for these purposes. Furthermore, the Attorneys General demand that X grant users control over whether their content can be edited by Grok, "including at a minimum the ability to easily prohibit the @Grok account from responding to their posts or editing their images when prompted by another user."
This push for greater transparency and expanded safeguards presents further challenges for X. Elon Musk, known for his "free speech" advocacy, has often argued that other apps offer similar functionalities and are not pursued with the same regulatory intensity. However, the Attorneys General directly addressed this point:
While other companies are also responsible for allowing NCII creation, xAI's size and market share make it a market leader in artificial intelligence. Unique among the major AI labs, you are connecting these tools directly to a social media platform with hundreds of millions of users. So your actions are of utmost importance. The steps you take to prevent and remove NCII will establish industry benchmarks to protect adults and children against harmful deepfake non-consensual intimate images.
The situation also highlights a notable contrast with Musk's previous public stance on Child Sexual Abuse Material (CSAM). Shortly after acquiring Twitter, Musk declared combating CSAM as "Priority #1" for the platform, criticizing previous leadership for their perceived failures. His current resistance to these demands, however, appears to contradict those earlier claims.
Ultimately, X is poised for increased scrutiny across multiple regions. The potential impacts could extend to xAI and Musk's wider artificial intelligence projects, significantly influencing his future plans. The industry awaits Musk's response and whether further legal or regulatory actions will be pursued.
