Regardless of whatever other augmented reality hardware solutions Apple might have been working on, the company’s immediate path forward appears to be iPhone-dependent AR glasses with a handheld controller, rather than a standalone platform. That’s the biggest surprise developers Steve Troughton-Smith and Guilherme Rambo have discovered in newly released versions of iOS 13 and XCode 11, which now show clear evidence of a stereo augmented reality feature called StarBoard — a necessary software prelude to the release of compatible hardware.
As of now, StarBoard looks similar to iOS’s CarPlay feature, which uses an iPhone to generate the graphics for a car’s external touch display. Unlike CarPlay, which outputs a single image for the 2D dashboard display, StarBoard appears to be focused on outputting two images for a 3D stereoscopic headset, which would be paired with a new touch-based controller for AR navigation. Code suggests that your hand will grasp something akin to a Siri Remote or Oculus Go controller, combining a touch surface and motion controls with a trigger and a Home button.
While it appears that Apple will have developers create objects and environments within its existing ARKit software environment, there are references in Apple’s code to Holokit, Garta, Franc, and Luck, which appear to be new AR software and hardware currently undergoing internal testing at the company. Troughton-Smith speculates that Apple might be priming a third-party accessory program to bring compatible glasses into the marketplace with its own offering as a “halo” product.
From a distance, Apple’s approach appears to be similar but not identical to Qualcomm’s XR Smart Viewer initiative, which will use Snapdragon 855-powered smartphones to process data and visuals for lightweight, screen-laden glasses. Qualcomm’s program was announced in February, shortly after Nreal demonstrated a 3-ounce pair of glasses called Light at CES that use a prior-generation Snapdragon 845 processor.
Priced at $500, Nreal Light occupies a 52-degree field of view with a bright, colorful 1080p resolution display that appears to make digital objects float plausibly in your field of view. Twin cameras and SLAM technology help an inside-out tracking system display the objects within real-world spaces, correctly adjusting their position and perspective as you move around them. Further advancements in graphics chips and display resolution will soon enable rendering of even more complex objects, likely approaching photorealism.
Two big challenges for Apple will be offering the accessory with both a form factor and a price that will resonate with consumers, rather than just enterprise users. Microsoft and Magic Leap have both released multi-thousand-dollar AR headsets that demand significant computing power, which Microsoft anchors on your head and Magic Leap tethers to your waist, using a cable, like Nreal Light. For numerous reasons, neither platform has taken off with mainstream audiences.
It remains to be seen whether Apple will be able to stream data wirelessly to the headset while offering enough battery power to make it usable for hours at a time. New iPhone 11 and iPhone 11 Pro models notably include both Wi-Fi 6 and an ultra-wideband wireless chip called U1, which could enable the company to steer fairly high-bandwidth wireless transmissions from iPhone to headset rather than relying on a cable.
Software will also be an issue, though a decision to rely on iPhones as the initial foundation for an AR headset could be canny. Apple could rely on developers to code iOS apps that will initially depend on iPhones, then spin off into apps that run directly on a more powerful future headset or over 5G from cloud servers. The company was originally said to be working on a custom variant of iOS called “rOS” for its wearable AR devices, which could emerge in the future as 5-nanometer chip manufacturing shrinks its A- or S-series processors down to glasses frame sizes.
Apple’s reveal timing for its headset remains unclear. A report earlier this year suggested that it had killed the headset project, while another claimed that it was mobilizing internal resources to finalize the project — possibly a sign that a more ambitious standalone device was shelved in favor of a more practical and shippable tethered one.