AI-generated audio and video platforms are rapidly advancing and finding applications in various domains, such as entertainment and HR. However, the leaders of these platforms acknowledge the associated risks. They understand the need to enforce moderation policies to prevent misuse, including the impersonation of public figures and fraudulent activities. During a discussion with Fortune senior writer Jeremy Kahn at the Fortune's Brainstorm AI conference in London, Victor Riparbelli, CEO of Synthesia, and Mati Staniszewski, CEO of ElevenLabs, highlighted the importance of ensuring that their companies' voice cloning and video generation technologies are used ethically. Riparbelli, one of the co-founders of the video-generation platform, stated, "As with most new tech, we immediately go to what things can go wrong. In this case, that’s right—this will be used by bad people to do bad things, that’s for sure. " Using AI-generated audio or video to impersonate public figures has become a contentious issue. Recent instances of explicit deepfakes involving Taylor Swift and fake Joe Biden robocalls have raised concerns about the influence of AI-generated content on the upcoming U. S. presidential election. OpenAI recently announced a delay in the rollout of its AI voice cloning tool due to the associated risks of misuse. They specifically cited concerns about political implications, stating, "We recognize that generating speech that resembles people’s voices has serious risks, which are especially top of mind in an election year. " Staniszewski, co-founder of ElevenLabs, emphasized their commitment to investing in know-your-customer protocols and mandatory disclosures. These measures are in place to ensure that all content generated on their platform can be traced back to a specific user's profile.

Additionally, the company is exploring ways to distinguish between AI-generated and authentic content, making it clear to users what is what. Riparbelli highlighted that Synthesia has established protocols that require users to verify their identity and provide consent before creating AI-generated videos. He stated, "It is impossible today to go in and take a YouTube video and make clones of someone [on Synthesia]. We take control that way. We have pretty heavy content moderation, rules about what you can create and what you cannot create. " During the discussion, a reporter raised concerns about audio deepfakes and referenced a viral audio clip impersonating London mayor Sadiq Khan, which criticized pro-Palestine marches in November last year. Khan expressed the need for action from Parliament to prevent bolder mischief makers. Staniszewski advocated for transparency, suggesting that all AI-generated content should be clearly labeled as such, and users should have tools to quickly verify the authenticity of messages. Riparbelli acknowledged that it will take time for the industry and lawmakers to reach a consensus on the best practices and regulations regarding the use of AI tools like those offered by Synthesia and ElevenLabs. He stated, "As with any new technology, you will have these years where people are figuring out what’s right and what’s wrong. "