If there’s a holy grail for augmented reality, developing a magic-quality way to instantly merge the real and digital worlds would be it — and a newly unearthed Apple acquisition could bring that one step closer to happening. Danish newspaper Børsen reports that Apple quietly spent approximately $30 million to purchase Spektral, a startup that uses machine learning to eliminate backgrounds from videos in real time on smartphones.
While the idea of using software techniques to separate people or objects from backgrounds is far from new — Apple added a similar feature to iChat back in 2006 as Backdrops — actually getting it to work well in real time is tricky. iChat Backdrops forced users to step out of the camera’s view and then superimposed whatever was different onto a fake background. Even at a low resolution, the effect always suffered from obvious pixel-level issues that made it clear the effect was artificial.
Before its acquisition, Spektral said it used “state-of-the-art machine learning and computer vision techniques” to “process images and video from the camera in real time (60 FPS) directly on the device.” Originally debuted under the name CloudCutout, its technology claims to intelligently separate people and objects as “cutouts,” seamlessly overlaying them on a new background. Demonstration videos evince what the company called “high quality details,” betraying no obvious sign of pixel-level issues.
The trick could have major implications for Apple’s AR initiatives. Being able to properly isolate specific elements in live video could enable ARKit applications where digital assets properly move in front of or behind real people, or where real people appear to be in fully or partially artificial surroundings. Like iChat Backdrops, it could also be used for cute special effects, akin to the company’s recent Animoji and Clips apps.
LinkedIn currently shows Spektral cofounder Toke Jansen as a San Francisco-based Apple manager of computational imaging, while another cofounder, Henrik Paltoft, continues to list one of his jobs as CEO of CloudCutout. As with all Apple acquisitions, it’s unclear when the acquired technology will make its way into shipping products, but the time frame after acquisitions like this has tended to be within two years.
VentureBeatVentureBeat's mission is to be a digital town square for technical decision-makers to gain knowledge about transformative technology and transact. Our site delivers essential information on data technologies and strategies to guide you as you lead your organizations. We invite you to become a member of our community, to access:
- up-to-date information on the subjects of interest to you
- our newsletters
- gated thought-leader content and discounted access to our prized events, such as Transform 2021: Learn More
- networking features, and more