SynergicLearning: Neural Network-Based Feature Extraction for Highly-Accurate Hyperdimensional Learning

TL;DR

This paper introduces a hybrid neural network and hyperdimensional learning model that combines high accuracy, quick training, and efficiency, suitable for on-line learning and hardware implementation.

Contribution

It presents a novel synergic model integrating neural networks with hyperdimensional classifiers, along with a hardware implementation and compiler for efficient on-chip learning.

Findings

Achieves neural network-level accuracy with improved hyperdimensional learning.

Enhances power efficiency by 1.60x and latency by 2.13x over existing HD models.

Supports incremental, on-line learning suitable for embedded applications.

Abstract

Machine learning models differ in terms of accuracy, computational/memory complexity, training time, and adaptability among other characteristics. For example, neural networks (NNs) are well-known for their high accuracy due to the quality of their automatic feature extraction while brain-inspired hyperdimensional (HD) learning models are famous for their quick training, computational efficiency, and adaptability. This work presents a hybrid, synergic machine learning model that excels at all the said characteristics and is suitable for incremental, on-line learning on a chip. The proposed model comprises an NN and a classifier. The NN acts as a feature extractor and is specifically trained to work well with the classifier that employs the HD computing framework. This work also presents a parameterized hardware implementation of the said feature extraction and classification components while introducing a compiler that maps any arbitrary NN and/or classifier to the aforementioned hardware. The proposed hybrid machine learning model has the same level of accuracy (i.e. $\pm$1%) as NNs while achieving at least 10% improvement in accuracy compared to HD learning models. Additionally, the end-to-end hardware realization of the hybrid model improves power efficiency by 1.60x compared to state-of-the-art, high-performance HD learning implementations while improving latency by 2.13x. These results have profound implications for the application of such synergic models in challenging cognitive tasks.