Robot Operating System (ROS), is one of the most robust and popular collections of robotics software frameworks on the planet. But its visualization tools are somewhat lacking, which spurred researchers at the Karlsruhe Institute of Technology to create an augmented reality (AR) companion app for mobile devices and heads-up displays like Microsoft’s HoloLens. Developed in Unity and dubbed Iviz, the open source app lets roboticists view a range of visualizations for ROS data, including point clouds and interactive markers.

In robotics, information recorded, generated, or analyzed by a robot is usually bound to a nearby physical position. Being able to visualize this data in situ can provide useful contextual clues that help guide the development of robots. Moreover, studies show that AR can shorten the time to complete certain tasks and reduce mistakes.

Iviz is cross-platform out of the box, with support for Windows, Linux, MacOS, iOS, and Android. A module enables AR visualizations of robots in simulation, which can be annotated with interactive markers representing things like target joint positions, objects to be grasped, and the moveable origins of a scene. This same module supports teleoperation courtesy of virtual joysticks, allowing an Iviz user to control a real-world robot and visualize data like lidar point clouds overlaid atop objects within the lidar sensor’s field of view.

ROS Iviz

Above: An AR visualization of terrain captured by a robotic excavator.

Broadly speaking, Iviz is built around the idea of displays, or reusable code that renders objects such as lines, point clouds, and duplicated meshes. As the researchers explain, this avoids the computational costs incurred by rendering many objects at once because instead of destroying and recreating objects, Iviz recycles discarded objects. If a frame axis made of cubes is no longer needed, for example, Iviz reuses the cube components as the links of a robot.

ROS Iviz

“There are many merits to having a mobile ROS platform. Using AR opens up the possibility for new, more intuitive forms of interaction with robots and the environment,” wrote the researchers in a paper describing Iviz’s design. “Furthermore, the Unity environment allows it to be easily ported to VR applications, which is the next step in our future work.”