Unless you have been living under a rock for the past few years, you know that things are really heating up in the present artificial intelligence (AI) arms race. Everyone seems to be trying to build more powerful processing hardware, or an even larger, smarter AI algorithm than what was released last week. But while these larger and faster technological advances capture most of our attention, that is not where all of the action is. Innovations in compact, energy-efficient hardware are also sorely needed.
The latest and greatest technology may have some very impressive capabilities, but it also consumes a great deal of power and relies on a lot of infrastructure to operate. As such, these systems run in large, remote data centers. And that can be a big issue for many applications, especially where maintaining privacy is a significant concern, or where real-time operation is necessary. For cases such as these, we need more efficient hardware that can run AI algorithms directly on-device.
A closer look at the chip (📷: W. Chao et al.)
A group led by researchers at the Technical University of Munich is working to develop a novel processing chip called the AI Pro that might help to fill a void in on-device AI processing. Unlike traditional chips that separate memory and computation units, the AI Pro integrates both functions closely using a method called hyperdimensional computing. This makes it possible for the chip to process information more like the human brain. In this way, it can recognize patterns and similarities without first crunching endless streams of raw data. And that dramatically reduces the amount of training data needed and slashes energy consumption.
For example, while conventional deep learning models require exposure to thousands or millions of images to learn what a car looks like, the AI Pro can infer this from a few pieces of semantic information. It might recognize that cars have four wheels, drive on roads, and come in certain basic shapes, for instance, and that would be enough to make a classification.
Architecture of the AI Pro (📷: W. Chao et al.)
The custom chip was fabricated with a 22nm process and features a RISC-V processor with vector extensions. It supports specialized fixed-point arithmetic that maintains accuracy with significantly lower complexity. A special VMAC-shift instruction enables fast vector operations by combining multiplication, accumulation, and shifting on multiple data points per clock cycle.
While the 10-million-transistor AI Pro cannot match the raw power of general-purpose AI chips like NVIDIA’s 200-billion-transistor behemoths, that is not the goal. The AI Pro is a niche chip aimed at doing a few, specific tasks, but doing them extremely well, and with minimal energy. And in a world where on-device and on-premises AI processing is growing increasingly important, this chip may fill an important role.