Epic Games, CubicMotion, 3Lateral, Tencent, and Vicon took a big step toward creating believable digital humans today with the debut of Siren, a demo of a woman rendered in real-time using Epic’s Unreal Engine 4 technology.
The move is a step toward transforming both films and games using digital humans who look and act like the real thing. The tech, shown off at Epic’s event at the Game Developers Conference in San Francisco, is available for licensing for game or film makers.
Cubic Motion’s computer vision technology empowered producers to conveniently and instantaneously create digital facial animation, saving the time and cost of digitally animating it by hand.
“Everything you saw was running in the Unreal Engine at 60 frames per second,” said Epic Games chief technology officer Kim Libreri, during a press briefing on Wednesday morning at GDC. “Creating believable digital characters that you can interact with and direct in real-time is one of the most exciting things that has happened in the computer graphics industry in recent years.”
Epic showed will be unveiling the Siren character, which was driven by a live actress. It showed realistic face and eye movement and the ability to interact with participants. The demo used 3Lateral’s facial rigging technology with body animation driven by Vicon’s motion capture system.
Cubic Motion’s computer vision technology tracks more than 200 facial features at over 90 frames per second. It automatically maps this data to high-quality digital characters in real-time.
“We are offering the keys to unlock a virtual world, enabling content producers and game developers to more easily interact with our technology and streamline the creation process for performance driven real-time digital humans,” said Andy Wood, Chairman of Cubic Motion, in a statement. “By 2020, this will no doubt transform content production across the board by making this technology universally available. By 2024, we may all be interacting with digital humans in some way or other, whether its via headsets, films, TV, games, live performances and broadcasts, or by directing digital assistants in our homes in real-time.”
Cubic Motion’s motion-capture technology allows a live actor to perform within the entertainment and game world in real-time. The technology transforms the production process, allowing directors and producers to see the result instantly and perform continuous retakes, saving time and money when developing games,film, TV and virtual assistants.
It also works for live esports broadcasts, allowing new content to be created in real-time and transmitted directly into a game.
The real-time presentation displayed during Epic’s keynote was previously recorded on Vicon’s capture stage at its headquarters in Oxford, England. To create the video, actress Alexa Leewore a full body motion capture suit with a head-mounted camera.
Using Vicon’s new Shōgun 1.2 software, her body and finger movements were captured on one screen while the data was streamed into Unreal Engine using Vicon’s new Live Link plugin. On a second screen, the Siren character — created using the likeness of Chinese actress Bingjie Jiang — moved in sync, driven in-engine at 60 frames per second.
“When we began working on Siren, we knew from the beginning that it was going to push several boundaries. To make this possible we needed the best motion capture hardware and software,” said Kim Liberi, Epic Games’ CTO.
The Siren project began as a collaboration between Epic and Tencent, with the goal of creating a proof-of-concept demonstration to show both the capabilities of Unreal Engine 4,and what the next generation of digital characters will look like.
Libreri said the real actress would demo the tech live at Epic’s booth during GDC.