NVIDIA has introduced an experimental generative AI model called the Foundational Generative Audio Transformer Opus 1, or Fugatto. This model, described as a "Swiss Army knife for sound, " uses text prompts to generate or modify audio, music, voice, and sound files. Designed by a global team of researchers, its "multi-accent and multilingual capabilities" have been enhanced, according to NVIDIA. Rafael Valle, a researcher and manager of applied audio research at NVIDIA, stated, "We wanted to create a model that understands and generates sound like humans do. " The company suggests that Fugatto could assist music producers in swiftly generating song prototypes, allowing easy edits for different styles, voices, and instruments. Fugatto could also be utilized to generate voice materials for language learning tools, and video game developers might use it to create variations of assets based on player actions. Furthermore, researchers discovered that with some fine-tuning, Fugatto can perform tasks beyond its pre-training, like combining separate instructions to generate specific speech or sound scenarios, such as a specific accent and emotional tone, or birds singing in a thunderstorm.

Additionally, it can produce sounds that evolve over time, such as a shifting rainstorm. NVIDIA has not confirmed public access to Fugatto. However, it is not the first generative AI capable of sound creation from text prompts. Meta has released an open-source AI kit for sound generation, and Google offers a text-to-music AI, MusicLM, available through its AI Test Kitchen website.