The money comes from 2Enable Partners and Hyperplane Venture Capital, and the Cambridge, Massachusetts-based company will use it to grow its team and bring the powers of voice customization to customers in gaming and social spaces.
“Visual skins” are in-game items which customize a character’s appearance, and they’ve been around for a very long time. They are the primary source of revenue for many games including Epic Games’ wildly successful Fortnite.
The analogous “voice skins” — which give those characters custom voices — have been a highly requested gaming feature. Modulate’s team has created true voice skins by utilizing adversarial deep neural networks alongside classical audio processing techniques. That lets players effectively customize their vocal cords while retaining full control over their emotion and inflection.
But Modulate’s founders believe their technology has an even greater significance than meeting this long-standing feature request.
“Online communities let us engage with people and ideas — or, in the case of video games, whole new worlds — without any preconceptions,” said Mike Pappas, CEO of Modulate, in a statement. “Until now, you were forced to use your real voice, shattering the freedom promised by these spaces and preventing some people from feeling safe using voice chat at all.”
He added, “Voice skins are fun to use, but what’s even more important is how they let you determine how you want to sound, which will allow more people to participate ever more deeply in these digital communities in the future. And of course, voice technology is expanding to every aspect of our lives, not just video games, so we foresee this technology having far-reaching effects across almost every industry as we continue to grow.”
John Huysmans, managing director at 2Enable Partners, was excited about Modulate’s potential to have a lasting impact on vocalization in gaming and other forms of media.
Modulate is not John’s first time investing in the space. He was also an early investor in Harmonix, the creators of Rock Band and Guitar Hero.
“The management team of Mike Pappas, Carter Huffman, and Terry Chen have continuously impressed us with their depth of understanding of the required technology, route to market, and its impact on the digital world,” said Huysmans, in a statement. “Their clear vision and desire to constantly seek advice and input from industry experts greatly enhances Modulate’s ability to succeed. Voice skins will undoubtedly create a new inflection point for how we interact in the digital world.”
Modulate’s voice skins will be available through select platforms via Modulate’s real-time software development kit (SDK). Users will be able to alter their voices nearly instantaneously. Some platforms will offer a variety of Modulate’s standard voices; others will choose to work with Modulate to design custom voices that precisely match a character or vision. Users will then have the freedom to swap between any of these voices or even mix and match them, designing a style that’s uniquely their own.
“Modulate is about creativity and freedom, not impersonating others,” said Modulate chief technology officer Carter Huffman, in a statement. “We’ve built ethical safeguards into our company from the ground up, from how we distribute our technology, to how we select the voice skins to offer, to watermarking our audio for detection in sensitive systems.”
Modulate’s key innovation is a novel application of deep learning style transfer, leveraging the potential of generative adversarial networks. Combining these techniques with a world-class understanding of audio signal processing allowed Modulate to overcome the limitations of either individual approach and produce high quality voice skins that sound completely natural. In addition, the voice skins work across different accents and even transcend languages, making this technology usable across the globe within a remarkably short period of time.
The founders were inspired by the success of visual style transfer techniques in apps such as Prisma. Applying these techniques to audio produced unintelligible results, but they eventually succeeded. Huffman conceived of Modulate in 2015 and incorporated it in the fall of 2017. Pappas also joined as founder, and Terry Chen, vice president of audio, also helped get the company off the ground.