Presented by Qualcomm Technologies, Inc.

Developers and companies are starting to see the major benefits of moving from centralized computing processes to decentralized ones as the cloud computing age approaches an end and edge computing takes center stage, says Jilei Hou, senior director of engineering at Qualcomm Technologies, Inc.

“One of the fundamental aspects of edge computing we’re working on is platform innovation, and how to offer the most efficient and effective processing tools to provide a scalable, supportive impact on the industry,” Hou says.

Qualcomm AI Research, an initiative of Qualcomm Technologies, Inc.,has an ambitious goal: to lead AI research and development across the whole spectrum of AI, particularly for on-device AI at the wireless edge. The company wants to be a vanguard in making on-device applications essentially ubiquitous.

The company has been focused on artificial intelligence for more than 10 years; when they launched their very first AI project for the company, they were part of the initial wave of companies recognizing the importance and potential of the technology. Next came inroads into deep learning, when they became one of the first companies looking at how to bring deep learning neural networks into a device context.

Currently Hou’s AI research team is doing a lot of fundamental research on the deep generative models that generate image, video, or audio samples, the generalized convolutional neural networks (CNN) to provide model equivariance against 2D and 3D rotation, and use cases like deep learning for graphics, computer vision, and sensor types beyond traditional microphones or cameras.

How edge computing will become ubiquitous

To usher in the age of edge computing and distribute AI into the devices, Qualcomm researchers are turning their attention to breaking down the obstacles on-device AI can present for developers, Hou says. In a relative sense, compared to cloud, there are very limited compute resources on-device, so processing is still confined by the area and the power constraints we have.

“In such a limited space, we still have to provide a great user experience, allowing the use cases to perform in real time in a very smooth manner,” he explains. “The challenge we face today boils down to power efficiency — making sure applications run well, while still staying under reasonable power envelope.”

Machine learning algorithms such as deep learning already use large amounts of energy, and edge devices are power-constrained in a way the cloud is not. The benchmark is quickly becoming how much processing can be squeezed out of every joule of energy.

Power-saving innovations

Qualcomm AI Research has also unlocked a number of innovations designed to allow developers to migrate workloads and use cases from the cloud to device in power-efficient ways, including the design of compact neural nets, how to prune or reduce the model size through model compression, compiling the model efficiently, and quantization.

“For example, Google is working on using machine learning techniques to enable search in the most efficient model architecture, and we’re doing a lot of exciting work trying to use similar machine learning techniques for model quantization, compression, and compilation in an automatic way,” says Hou.

A lot of app developers, or even researchers in the community today are only aware or focused on the floating point models, Hou continues, but what his team is thinking about is how to transform floating point models into quantization, or fixed point models, which makes a tremendous impact on power consumption.

“Quantization may sound simple to a lot of folks,” Hou says. “You simply convert a floating to a fixed point model. But once you try to convert to fixed point models, in very low bit width — eight bits, four bits, or potentially binary models – then you realize there’s a great challenge, and also design tradeoffs.”

With post-training quantization techniques, where you do not rely on model retraining, or in a situation where the bit width becomes very low, going to binary models, how can you even preserve the model’s performance or accuracy with the fine tuning allowed?

“We are now in the most convenient position to conduct system hardware co-design, to make sure we provide tools to help our customers efficiently convert their models to low bit width fixed point models, and allow very efficient model execution on device,” he explains. “This is definitely a game changing aspect.”

Qualcomm AI research use cases

“We’re focused on providing the quantization, compression, and compilation tools to make sure researchers have a convenient way to run models on device,” Hou says.

The company developed the Qualcomm Snapdragon Mobile Platform to enable OEMs to build smartphones and apps that deliver immersive experiences. It features the Qualcomm AI Engine, which makes compelling on-device AI experiences possible in areas such as the camera, extended battery life, audio, security, and gaming, with hardware that helps ensure better overall AI performance, regardless of a network connection.

That’s been leading to some major innovations in the edge computing space. Here are just a few examples.

Advances in personalization. Voice is a transformative user interface (UI) – hands-free, always-on, conversational, personalized, and private. And there are a huge chain of real-time events required for on-device AI-powered voice UI, but one of the most important might be user verification, Hou says, meaning the voice UI can recognize who is speaking and then completely personalize its responses and actions.

User verification is particularly complex because every human’s voice, from sound to pitch to tone, changes in response to season changes, temperature changes, or even just moisture in the air. To achieve the best performance possible requires the advances in continuous learning that Qualcomm Technologies’ researchers are making, which lets the model itself adapt to changes in the user’s voice over time.

As the technology matures, emotion analysis is also becoming possible, and researchers are looking for new ways to design and incorporate those capabilities and features into voice UI offerings.

Efficient learning leaps. Convolutional neural nets, or CNN models, can handle what’s called a shift invariance property, or in other words, any time a dog appears in an image, the AI should recognize it as a dog, even if it’s horizontally or vertically shifted. However, the CNN model struggles with rotational invariance. If the image of the dog is rotated 30 or 50 degrees, the CNN model performance will degrade quite visibly.

“How developers deal with that today is through a workaround, adding a lot of data augmentation, or adding more rotated figures,” Hou says. “We’re trying to allow the model itself to have what we call an equivariance capability, so that it can handle image or object detection in both a 2D or 3D space with very high accuracy.”

Recently researchers have extended this model to any arbitrary manifolds, applying the mathematical tools coming out of relativity theory from the modern physics field, he adds, using similar techniques to design equivariance CNN in a very effective way. The equivariance CNN is also a fundamental theoretical framework that enables more effective geometric deep learning in 3D space, in order to recognize and interact with objects that have arbitrary surfaces.

The unified architecture approach. In order for on-device AI to be efficient, neural networks have to become more efficient, and unified architecture is the key. For example, even though audio and voice come through the same sensor, a number of different tasks might be required, such as classification which deals with speech recognition; regression, for cleaning up noise from audio in order to be further processed; and compression, which happens on a voice call, with speech encoding, compression, and then decompression on the other side.

But though classification, regression, and compression are separate tasks, a common neural net can be developed to handle all audio and speech functions together in a general context.

“This can help us in terms of data efficiency in general, and it also allows the model to be really robust across different tasks,” Hou says. “It’s one of the angles we’re actively looking into.”

Research obstacles

The obstacles researchers face in general fall into two categories, Hou says.

First, researchers must have the best platform or tools that can be available to them, so they can conduct their research or port their models to the device, making sure they can have a high-quality user experience from a prototyping perspective.

“The other comes down to fundamentally marching down their own research path, looking at the innovation challenges and how they’re going to conduct research,” Hou says. “For machine learning technology itself, we have a really good challenge, but the opportunities lie ahead of us.”

Model prediction and reasoning is still in its early stage, but research is making strides. And as ONNX becomes more widely adopted into the mobile ecosystem, model generalizability gets more powerful, object multitasking gets more sophisticated, and the possibilities for edge computing will continue to grow.

“It’s about driving AI innovation to enable on-device AI use cases, and proactively extend leveraging 5G to connect the edge and cloud altogether, where we can have flexible hybrid training or inference frameworks,” Hou says. “In that way we can best serve the mobile industry and serve the ecosystem.”

Content sponsored by Qualcomm Technologies, Inc. Qualcomm Snapdragon is a product of Qualcomm Technologies, Inc. and/or its subsidiaries.

Sponsored articles are content produced by a company that is either paying for the post or has a business relationship with VentureBeat, and they’re always clearly marked. Content produced by our editorial team is never influenced by advertisers or sponsors in any way. For more information, contact