AI-powered sound generation is advancing at a scarily rapid pace: just a few years ago, you might have laughed at the mere prospect of an AI model generating a fully-fledged pop song complete with realistic vocals, but here we are.
The latest tech giant to enter the fray is chip manufacturer Nvidia, who has designed a generative AI model that it says has the power to create not only music, but also to generate sounds that you've "never heard before".
Described by Nvidia as a "Swiss Army knife for sound", Fugatto (an abbreviation of Foundational Generative Audio Transformer Opus 1) is said to be able to create "any combination of music, voices and sounds". "Whatever users can describe, the model can create," reads a post on Nvidia's website.
Much like AI music generators such as Suno and Stable Audio, Fugatto can generate musical clips based on prompts provided by the user, edit existing songs by removing or adding instruments, and isolate instrumental and vocal stems. Where it differs, Nvidia says, is its ability to "explore entirely new realms, producing sounds that bring creative concepts to life... opening up new possibilities for creativity and production."
In the video embedded above, the user asks Fugatto to "create a saxophone howling, barking then electronic music with dogs barking". The resulting audio sounds, admittedly, a great deal like a hybrid of a free jazz sax solo and a displeased canine. Whether this sound would be useful to anyone in a musical context remains to be seen.
Nvidia suggests that music producers could use Fugatto to experiment with initial ideas for songs, testing out different styles, instruments or voices, or even applying audio effects.
In the video above, a synth line is uploaded to Fugatto before the users instructs the AI model to augment it with a drum part. In the following example, a simple piano melody is given to Fugatto with the instruction: "turn this MIDI melody into a female voice, operatic scat singing style". The results are impressive, if a little lo-fi.
“The history of music is also a history of technology," says producer Ido Zmishlany, a member of Nvidias' Inception program. "The electric guitar gave the world rock and roll. When the sampler showed up, hip-hop was born. With AI, we’re writing the next chapter of music. We have a new instrument, a new tool for making music — and that’s super exciting.”
Nvidia hasn't said whether it intends to make Fugatto available to the public.
