Interested in learning what's next for the gaming industry? Join gaming executives to discuss emerging parts of the industry this October at GamesBeat Summit Next. Register today.

When companies release new augmented reality glasses, the key details most people focus on are the size and resolution of the AR displays within users’ field of view. The larger and more detailed a headset’s augmented “window,” the better. But Swiss medical researchers are now spotlighting 3D tracking performance as an underappreciated specification when enabling AR use in critical procedures, such as surgery.

According to a group of researchers from the University of Zurich, ETH Zurich, and Balgrist University Hospital, AR headsets such as Microsoft’s HoloLens could be used to improve spinal fusion surgery, allowing surgeons to place digitally generated 3D anatomical maps on a patient’s surgical site rather than using invasive radiation to identify a location. The key problems, however, are that the first HoloLens wasn’t built for high-fidelity 3D tracking, and it’s subject to hologram drifting and application instability even if the wearer attempts to minimize head movement.

Seamlessly combining the real and digital worlds has been the end goal of augmented reality hardware developers for years, but actually achieving the feat has proved tricky, and most systems merely overlay digital objects on top of real ones, rather than “occluding” digital objects with real items in the foreground. As demanding as this might be when adding a virtual pillow to a real sofa, getting the millimeter-accurate positioning right so a doctor can insert a spinal fusion screw is another level of challenge.

The Swiss researchers tested their system by collecting 3D mapping point clouds of bone surface regions from realistic but “phantom” mockups of spines, rather than using human subjects. Notably, the phantoms were rigidly attached to a table to keep them from moving during the tests.


GamesBeat Summit Next 2022

Join gaming leaders live this October 25-26 in San Francisco to examine the next big opportunities within the gaming industry.

Register Here

To use HoloLens for the simulated procedure, the researchers needed to activate Research Mode to access the headset’s two front environmental tracking cameras, which operate at only 640 x 480 resolution, and then use custom pointing devices to mark and digitally “register” surgical locations on vertebrae. Surgeons then used the marked locations and virtual imagery to properly bend and insert rods for spinal fusion. In trials with the phantom spines, the researchers found both timing and accuracy of the AR system to be comparable to state-of-the-art non-AR alternatives.

That said, the researchers are hoping for improved software and hardware that will eliminate hologram drift and provide a more stable application environment during high-precision 6DoF tracking — features that might give future developers of enterprise-class AR solutions some guidance as to how to differentiate their glasses from far less expensive consumer models that are coming to market this year. In the future, the Swiss researchers plan to use true depth-sensing cameras, such as Microsoft’s Azure Kinect, to improve accuracy and automation of the process for surgical needs.

VentureBeat's mission is to be a digital town square for technical decision-makers to gain knowledge about transformative enterprise technology and transact. Discover our Briefings.