THDC: Training Hyperdimensional Computing Models with Backpropagation

TL;DR

THDC introduces a trainable hyperdimensional computing framework using backpropagation, enabling end-to-end learning with reduced dimensionality and improved efficiency for resource-constrained devices.

Contribution

It replaces static hypervectors with trainable embeddings and incorporates a neural network for optimized class representations, advancing HDC capabilities.

Findings

Achieves comparable or better accuracy than state-of-the-art HDC.

Reduces hypervector dimensionality from 10,000 to 64.

Demonstrates effectiveness on MNIST, Fashion-MNIST, and CIFAR-10.

Abstract

Hyperdimensional computing (HDC) offers lightweight learning for energy-constrained devices by encoding data into high-dimensional vectors. However, its reliance on ultra-high dimensionality and static, randomly initialized hypervectors limits memory efficiency and learning capacity. Therefore, we propose Trainable Hyperdimensional Computing (THDC), which enables end-to-end HDC via backpropagation. THDC replaces randomly initialized vectors with trainable embeddings and introduces a one-layer binary neural network to optimize class representations. Evaluated on MNIST, Fashion-MNIST and CIFAR-10, THDC achieves equal or better accuracy than state-of-the-art HDC, with dimensionality reduced from 10.000 to 64.