Qualcomm touts eight AI ‘firsts’

Qualcomm Technologies’ AI research group has pioneered innovations ranging from on-device learning to wireless AI, and has ambitious plans for the next set of “firsts.”

Whenever people take photos or talk to a digital assistant using a mobile phone, they often don’t realize they’ve just taken advantage of artificial intelligence (AI). If they think of AI, it’s usually in the context of self-driving vehicles or perhaps Facebook’s massive data centers (Meta). As AI becomes ubiquitous and distributed across edge devices and cloud servers, many challenges remain to realize CEO Cristiano Amon’s Connected Intelligent vision for AI to enable automated perception, reasoning and action.

For AI to enable levels of automation and personalization, Jilei Hou, vice president of Qualcomm AI Research, believes that AI hardware and software must become much smaller, faster, more efficient, less power-hungry energized and able to continuously learn at the forefront of the real world. This provides the perfect complement to cloud-based remote processing, the reach of which has been further enhanced with Qualcomm’s 5G technology. These aren’t just engineering challenges; progress and breakthroughs must come from basic, applied and platform science research.

First AIs from Qualcomm AI Research

Qualcomm AI Research recently hosted a webinar where Jilei Hou, VP of Engineering and Head of AI R&D at Qualcomm AI Research, outlined “firsts” in eight research areas they are justifiably proud of. Their new AI research and comprehensive AI optimizations pushed the AI ​​industry forward and enabled the first-ever proof-of-concept demonstrations on commercial mobile devices.

Their firsts in AI include 8-bit model quantification leading to the industry’s best energy efficiency toolkit (the AI ​​Model Efficiency Toolkit, or AIMET), on-device learning demonstrating a 30% improvement in keyword detection, federated learning with an end-to-end software framework, real-time video semantic segmentation, group equivariant CNNs, AI for wireless with weakly first method supervised for passive RF sensing, super-resolution video at 4K at 100 FPS on mobile and neural video compression with the first real-time HD decoding on mobile. We interviewed Hou recently and learned a lot about mobile AI challenges and solutions.

Artificial intelligence at Qualcomm, like almost everything else in the company, starts with mobile. And that legacy sets the bar high, since mobile devices are lightweight, battery-powered consumer devices with limited processing, memory, and I/O. Ongoing research at Qualcomm AI Research focuses on three main areas: basic research, applied research, and platform research. The latter covers topics such as energy efficiency and on-device learning, which are essential in mobile but have also benefited Qualcomm in other verticals like IOT, XR and automotive. But efficiency doesn’t stop at hardware; Qualcomm AI Research is also advancing model design, compression, quantization, algorithms and software tools to tackle the problem through comprehensive research.

The efficiency comes partly from using less data in the model and from quantification. If one can run a model with 4-bit integer math, it will be up to 64 times more efficient than using 32-bit floating point. But all this efficiency cannot be achieved at the expense of reduced precision. At this point, Qualcomm AI Research has demonstrated a state-of-the-art 8-bit transformer model with less than 1% accuracy degradation.

Today, some of the AI ​​processing needs to be offloaded from the mobile device to a cloud service. However, faster response times, better privacy, better personalization, and better understanding in the context of the request will require on-device learning.

To look forward

Qualcomm AI Researchers are pursuing additional new “firsts” that could translate into significant value and differentiation for customers and partners. Targeting AI efficiency improvements to mobile devices and the rest of the connected intelligent edge can pave the way for improved customization and automation of many tasks, enabling new use cases and enhancing the experience global user.


The increase in AI processing on edge devices is both inevitable and extremely valuable for improving user experience and device functionality. The challenges remain daunting, given the limited amount of processing capacity and memory on mobile platforms. Qualcomm Technologies Inc is rising to the challenge, and these 8 “AI Firsts” demonstrate that the organization is already making significant progress, with more innovations in sight.

Disclosures: This article expresses the opinions of the author and should not be considered advice on buying or investing in the companies mentioned. My company, Cambrian AI Research, is fortunate to have many, if not most, semiconductor companies as customers, including Blaize, Cerebras, Esperanto, Graphcore, IBM, Intel, NVIDIA, Qualcomm Technologies, Synopsys, and Tenstorrent. We have no investment position in any of the companies mentioned in this article and do not plan to initiate any in the near future. For more information, please visit our website at https://cambrian-AI.com.

Comments are closed.