DecoHD: Decomposed Hyperdimensional Classification under Extreme Memory Budgets

TL;DR

DecoHD introduces a learned, decomposed hyperdimensional computing method that significantly reduces memory and energy consumption while maintaining high accuracy and robustness, suitable for deployment on resource-constrained devices.

Contribution

It presents a novel end-to-end trainable decomposed HDC framework that compresses class prototypes efficiently without sacrificing native HDC properties.

Findings

Achieves up to 97% fewer trainable parameters with minimal accuracy loss.

Provides substantial energy and speed improvements over CPUs, GPUs, and baseline HDC hardware.

Maintains within 0.15% accuracy of strong non-reduced HDC baseline.

Abstract

Decomposition is a proven way to shrink deep networks without changing input-output dimensionality or interface semantics. We bring this idea to hyperdimensional computing (HDC), where footprint cuts usually shrink the feature axis and erode concentration and robustness. Prior HDC decompositions decode via fixed atomic hypervectors, which are ill-suited for compressing learned class prototypes. We introduce DecoHD, which learns directly in a decomposed HDC parameterization: a small, shared set of per-layer channels with multiplicative binding across layers and bundling at the end, yielding a large representational space from compact factors. DecoHD compresses along the class axis via a lightweight bundling head while preserving native bind-bundle-score; training is end-to-end, and inference remains pure HDC, aligning with in/near-memory accelerators. In evaluation, DecoHD attains extreme memory savings with only minor accuracy degradation under tight deployment budgets. On average it stays within about 0.1-0.15% of a strong non-reduced HDC baseline (worst case 5.7%), is more robust to random bit-flip noise, reaches its accuracy plateau with up to ~97% fewer trainable parameters, and--in hardware--delivers roughly 277x/35x energy/speed gains over a CPU (AMD Ryzen 9 9950X), 13.5x/3.7x over a GPU (NVIDIA RTX 4090), and 2.0x/2.4x over a baseline HDC ASIC.