Augmented reality helps you fix your printer

Augmented reality (AR) company Metaio will show a demo at next week’s Mobile World Congress in which AR on your mobile phone shows you how to change a printer toner cartridge.

Augmented reality overlays digital information over your view of the physical world in a mobile device’s camera view

The printer demo is significant for a couple of reasons: the application of mobile AR to a service and maintenance scenario, and the sophisticated technology used.

Metaio has a division which works with industrial clients like the car company Volkswagen. For example, a 3D model of a new vehicle can be superimposed over a digital photo of the actual factory environment to help with plan the assembly room layout. This kind of application requires very precise 3D modeling and overlay of the AR image over the real-world image.

For the printer demo, Metaio’s software creates a virtual 3D point cloud (see the video) from the image of the printer in the mobile phone’s camera view. This point cloud defines the structure of the printer. The cloud is then compared to a markerless tracking reference (a sort of “signature” created from 3D CAD models of the printer) so the software can recognize the correct printer model. The software then renders AR objects over the camera view of the printer in real time and with the correct scale in order to instruct the user on how to change the cartridge.

Most mobile AR applications so far have been simple, sometimes gimmicky and according to some commentators not even real AR. Now that the initial novelty has worn off, Metaio is convinced that AR users are looking for more useful applications and, in areas like games and media, higher production values. Replacing the service manual certainly falls into the category of useful.

Improving AR graphics for applications like games is made difficult by the already high processing and battery power required by AR. Metaio worked with ST-Ericsson’s latest dual-core smartphone platform, the U8500, to optimize its AR software to run on that chip set (the printer demo can be seen in the in the ST-Ericsson partner zone at Mobile World Congress).

This increased the frames per second of the AR image rendering from about 18 frames/second on an off-the-shelf smartphone to 30 frames/second on ST Ericsson’s platform, resulting in a much smoother and more accurate AR experience.The company thinks that such partnerships between hardware and AR software vendors are essential to push mobile AR technology forward.

Although the functionality shown in the printer demo is not yet available in Metaio’s mobile AR browser Junaio, the company has been busy adding other interesting features like image recognition and visual search (similar to Google Goggles) including face recognition.

Metaio is based in Munich, Germany, was founded in 2003 and has around 65 employees. It is privately funded.