Arm’s Mali-D77 targets barf-free, high-performance, untethered VR headsets

Arm is unveiling its design for the Arm Mali-D77 display processor, which is targeted at next-generation virtual reality head-mounted displays.

Arm’s client line of business vice president Nandan Nayampally said in a blog post that the new processor can help eliminate motion sickness. If a processor alone can really do that, I’ll be really surprised. But that’s the way they are marketing it.

The new processor can achieve more than 40% system bandwidth savings and 12% power savings for VR workloads.

VR definitely needs the help. While enterprise VR is doing well, consumer sales of VR headsets are tepid, thanks to the high price, the tethered experience, and the complexity of the systems.

And Arm said the new chip enables development of smaller, lighter, more comfortable untethered VR devices, scaling from HMDs to standard premium mobile displays.

Arm agrees that users want to see greater immersion, smoother performance, and untethered and lighter devices. However, developing the hardware to match the performance levels, while keeping the efficiency for untethered operation to deliver the VR experiences that consumers are craving remains a challenge.

Nayampally said that the heart of this challenge is the display, which sets the tone for the VR performance levels on devices, from the quality of the visuals to latency (or interaction delays).

The Mali-D77 is the solution “to surpass the proverbial last mile in the journey toward untethered immersive experiences,” he said.

Less virtual, more reality with all new VR functionality

Above: The design of the Arm Mali-D77.

Image Credit: Arm

The Mali-D77 enables offloading of specific compute functions from the graphics processing unit (GPU) to the display processing unit (DPU), leading to higher-quality visuals and eliminating motion sickness which in turn frees up more GPU cycles and associated system bandwidth.

Other Mali-D77 enhanced capabilities include:

• Lens Distortion Correction (LDC): predistorts the images to counter the effect of the lens so that when they are viewed through the lenses of any VR headset they appear correct and undistorted.
• Chromatic Aberration Correction (CAC): pre-separates the color channels in the opposite direction in order to counteract the blurring effect caused by the lenses of the VR headset.
• Asynchronous Timewarp (ATW): translates and re-projects the virtual scene based on the latest head pose and position of the headset in the 3D space.

A versatile display technology

Above: Arm Mali-D77 is a display processor aimed at better VR.

Image Credit: Arm

Beyond VR head-mounted displays (HMDs), the versatile Mali-D77 brings benefits to other devices, displays and use cases. For example, Mali-D77 can be integrated into a common system-on-a-chip (SoC) platform with existing developer ecosystems for switching across multiple devices such as a VR HMD to LCD/OLED large or small screens capable of displaying 4K HDR scenes. By doing this, VR devices can become more affordable and accessible to consumers.

Today’s state-of-the-art VR devices can display 2,880-by-1,440 pixels, but the trend is moving towards increased resolutions within ever-shrinking power budgets on lighter, more comfortable headsets.

The bar is already high and expected to be raised even further, with VR HMDs capable of 2.5K-by-2.5K — or even 3K-by-3K per eye — likely to be in production by major panel vendors in 2021.

Looking further ahead into the future, this is set to increase towards a whopping 4K-by-4K per eye where there will be a complete absence of any visual artifacts, Arm said.

In addition to increased pixel resolution, higher frame rates of up to 120 frames per second (fps) are needed to reduce the overall latency to prevent motion sickness and achieve low persistence on LCD VR panels.

Also, for a more immersive experience, there is a need for a higher the Field of View (FOV), great than 100 degrees, to achieve the feeling of presence in the scene. This is set to increase to around 140 degrees by 2021.

An SoC that can drive this level of performance for wearable VR HMDs does not exist today. This presents a major challenge to SoC vendors who need to achieve the above requirements — and more — in less than 3 watts, Arm said.

Mali-D77 is a significant step to addressing these VR HMD design challenges, the company said.

Crucially, Mali-D77 will also eliminate motion sickness through the higher frame rates and responding to real-world head movements by re-projecting the VR scene according to the latest pose due to its “just-in-time” single pass composition process. Moreover, it will deliver crisper images free from any artifacts through the more advanced hardware-based filtering and image processing functions it possesses.

When you’re viewing using the standard smartphone display, Mali-D77 will operate in smartphone mode — effectively becoming identical in operation to the Mali-D71 DPU.

Above: Offloading from the GPU to the DPU could solve VR’s problems.

Image Credit: Arm

In smartphone mode Mali-D77 can drive up to 8 video/graphics composition layers at a performance of 4K60 or 4 video/ graphics composition layers at performance of 4K120fps (when driven side-by-side).

When you switch to the VR HMD, Mali-D77 can render a VR scene composed of up to 4 stereo VR layers (layers subject to ATW/LDC/CAC) and 4 normal video/graphics layers at  2,880-by-1,440 pixels.

At 4K90fps, Mali-D77 can drive up to 2 stereo VR layers and 2 normal layers.

“The Mali-D77 DPU is exactly the product needed in the market to drive higher resolutions (over 1000ppi) VR displays,” said Synaptics’ Jeff Lukanc, senior director of marketing, in a statement. “Arm and Synaptics are in an early collaboration phase to enable an optimal solution for foveated transport across Mali-GPU, Mali-D77 and Synaptics’ R63455 VR display solution with the ultimate aim of driving higher resolution without increasing the overall cost for future VR devices.”