Our world has become increasingly interconnected, granting individuals access to a vast amount of multilingual content. With this heightened connectivity comes the growing importance of effective communication and comprehension of information in various languages. Today, we are proud to introduce SeamlessM4T, an innovative all-in-one AI translation model. This groundbreaking technology enables effortless communication through speech and text across different languages. SeamlessM4T provides support for this transformative experience. As part of our commitment to open science, we are releasing SeamlessM4T under a research license, allowing researchers and developers to build upon this remarkable advancement. We are also making the metadata of SeamlessAlign available—a vast open multimodal translation dataset consisting of an impressive 270, 000 hours of meticulously mined speech and text alignments. The development of a universal language translator, akin to the fictional Babel Fish from The Hitchhiker's Guide to the Galaxy, is a formidable challenge. Existing speech-to-speech and speech-to-text systems only cover a small fraction of the world's languages. However, we firmly believe that the announcement we are making today represents a substantial leap forward in this mission. Compared to traditional approaches that rely on separate models, the single system approach of SeamlessM4T significantly reduces errors and delays, enhancing the efficiency and quality of the translation process.

This empowers individuals who speak different languages to communicate with each other more effectively. SeamlessM4T builds upon the progress we and others have achieved over the years in our quest to develop a universal translator. Last year, we unveiled No Language Left Behind (NLLB), a text-to-text machine translation model supporting 200 languages. NLLB has since been integrated into Wikipedia as one of the translation providers. In addition, we showcased a demo of our Universal Speech Translator, which marked the world's first direct speech-to-speech translation system for Hokkien—a language without a widely used writing system. Earlier this year, we also unveiled Massively Multilingual Speech, a technology that offers speech recognition, language identification, and speech synthesis capabilities across over 1, 100 languages. SeamlessM4T builds upon the insights gathered from these projects, resulting in a remarkable multimodal and multilingual translation experience derived from a single model. This cutting-edge technology is built upon an extensive range of spoken data sources and produces state-of-the-art results. Today's announcement represents just the latest stride in our ongoing commitment to creating AI-powered technology that connects people across different languages. Moving forward, we aim to explore how this foundational model can facilitate new communication capabilities, ultimately bringing us closer to a world where mutual understanding is universal. For more information on SeamlessM4T, please visit our AI blog.