Hundreds of AI Apps Hidden Sexually Abusive Deepfake Features
Hundreds of AI apps with hidden sexual deepfake features were found on major app stores.
"Seriously, 400 apps hiding deepfake abuse? This isn't just a glitch, it's a massive policy fail. Time for app stores to step up their game."
A network of over 400 AI apps and websites, linked to Chinese-language developers, were discovered to contain secret features for generating violent and sexually abusive deepfake images and videos. These apps, often disguised as mundane tools like pet portrait generators or workout coaches, bypassed checks on Apple and Google app stores. The investigation, primarily by WIRED and other researchers, revealed that these apps would stealthily load explicit web interfaces under certain conditions, allowing users to create deepfake content.
The network appears to be run by Chinese developers and a Hong Kong-registered entity. Researchers traced the apps after investigating thousands of video ads containing explicit deepfake content, including AI-generated child sexual abuse material, published on Meta's Facebook and Instagram. These ads linked directly to the deceptive apps, enabling users to generate explicit images and make payments using Apple and Google Pay.
This discovery highlights a significant challenge for app store policy enforcement, as the network has reportedly been publishing these apps for approximately a year, sometimes dozens per week. The hidden behavior of these apps, which determine whether to show benign or explicit content based on user information, allowed them to evade detection despite stated policies against harmful deepfake content.
This incident underscores critical vulnerabilities in app store content moderation and the potential for AI tools to be exploited for harmful purposes. Businesses relying on app store distribution must be aware of the sophisticated methods used to bypass security, impacting brand trust and user safety.
Relevant tools
Find the right AI tool for your business
Chat with Insta and get matched to the right tool in seconds.
Try Insta Tool Finder →