San Francisco City Attorney David Chiu has issued a formal demand to Apple and Google to remove dozens of "nudify" applications from their respective app stores. These AI-powered tools digitally alter photographs to depict clothed individuals as nude without their consent. Both tech giants have been granted 28 days to address the issue or face potential civil penalties under California law regarding the facilitation of non-consensual deepfake content.
Regulatory Pressure and Legal Standing
The demand from the San Francisco City Attorney’s office rests on both existing statutes and upcoming legislation. California law currently criminalizes the knowing facilitation or reckless aiding of non-consensual deepfake pornography. Furthermore, a new law set to take effect in 2025 will empower victims to pursue civil litigation against third-party facilitators who host or distribute such content.
According to statements provided to TechCrunch, David Chiu emphasized that Apple and Google have profited from these applications for nearly a year. Chiu noted that the companies have likely collected millions of dollars in transaction fees from the distribution of these services. The City Attorney’s office argues that despite some removals, the platforms must adopt a more proactive stance to prevent the exploitation of women and girls.
Platform Responses and Enforcement Actions
Both Apple and Google have acknowledged the receipt of the letters and reported initial steps toward enforcement.
- Apple: A spokesperson for the company stated that "nudify" apps violate existing App Store policies. The company confirmed the removal of three specific applications mentioned in the correspondence and is currently in the process of terminating the associated developer accounts. Apple is also reviewing four additional apps for potential policy violations.
- Google: The company reported that all five applications referenced in the City Attorney’s letter have been suspended from the Google Play Store. Google stated it has previously taken action against hundreds of similar apps and has implemented restrictions on search terms like "nudify" to curb discoverability.
Neither company has publicly disclosed the total revenue generated from these specific apps or the total number of downloads facilitated through their platforms.
Origins of the Investigation
The City Attorney’s action follows two investigative reports published by the Tech Transparency Project in January and April of this year. These reports identified dozens of applications on both platforms that allowed users to generate non-consensual intimate imagery in exchange for payments. The findings alleged that both Apple and Google actively guided users toward these tools, effectively acting as key intermediaries in the proliferation of AI-generated sexualized content.
Reporting and Protective Measures
Users who encounter applications that generate non-consensual imagery can utilize built-in reporting features to flag them for review:
- Apple App Store: Users can select the "Report a Problem" option located on the individual app listing page.
- Google Play Store: Users can select the "Flag as inappropriate" option, which is located under the developer contact section of the app page.
When reporting these apps, it is recommended to specify that the content relates to the generation of non-consensual intimate imagery. While Apple and Google have committed to investigating reports, the long-term impact on developers remains tied to platform policy enforcement. Developers creating generative AI tools for photo editing must ensure their software includes safeguards against non-consensual use to avoid permanent removal from these ecosystems.