Muah. AI is a platform that allows users to create AI chatbots as virtual girlfriends, enabling text, voice conversations, and image sharing. Almost 2 million users are registered on the site, which claims to offer "uncensored" technology. However, recent reports indicate that some individuals may be using the platform to generate child sexual abuse material (CSAM). Investigative journalist Joseph Cox from 404 Media revealed that an anonymous hacker leaked disturbing data from Muah. AI, showing prompts involving explicit content related to minors, including alarming requests specifying newborns. Unlike major AI platforms like ChatGPT that implement strict content filters, smaller services like Muah. AI appear to have fewer safeguards. Troy Hunt, a security expert who reviewed the leaked data, discovered numerous requests for CSAM, with searches for terms like "13-year-old" yielding over 30, 000 results. Hunt noted that some users revealed their identities without apparent concern, raising questions about the prevalence of such disturbing content on mainstream sites. In a conversation about the hack, a Muah. AI representative acknowledged the breach but disputed claims regarding the volume of abusive prompts, arguing that such a high percentage of users couldn’t be engaged in pedophilia.

The staff, limited to fewer than five members, lacks adequate resources for monitoring user activity, although they have implemented some filtering measures that were criticized by users for being overly restrictive. While Han, representing Muah. AI, expressed an intention to improve monitoring, he admitted that the company doesn’t actively verify if CSAM is being generated. He suggested that requests for such images might originate from users grieving the loss of loved ones, although he recognized that some users might bypass the filters. The discussion highlighted broader concerns about the existence and regulation of AI platforms that may unwittingly facilitate abuse. Han argued for freedom of speech in technology, comparing his service to companies like Google, which provide processing without direct oversight of content. Legal ambiguities regarding AI-generated imagery, particularly CSAM, remain contentious, especially after past Supreme Court rulings. The existence of platforms like Muah. AI reveals significant issues in the realm of generative AI, suggesting that the creation of exploitative material is a growing problem that is difficult to control or eliminate, with concerns about the misuse of such technologies likely to continue.