HD-CB: The First Exploration of Hyperdimensional Computing for Contextual Bandits Problems

TL;DR

This paper introduces Hyperdimensional Contextual Bandits (HD-CB), a novel approach using hyperdimensional computing to efficiently solve sequential decision-making problems with competitive performance and faster convergence.

Contribution

It pioneers the application of hyperdimensional computing to model and solve contextual bandit problems, replacing complex regression with simple vector operations.

Findings

HD-CB achieves competitive or superior performance compared to traditional methods.

HD-CB offers faster convergence and lower computational complexity.

HD-CB demonstrates high scalability and parallelism in experiments.

Abstract

Hyperdimensional Computing (HDC), also known as Vector Symbolic Architectures, is a computing paradigm that combines the strengths of symbolic reasoning with the efficiency and scalability of distributed connectionist models in artificial intelligence. HDC has recently emerged as a promising alternative for performing learning tasks in resource-constrained environments thanks to its energy and computational efficiency, inherent parallelism, and resilience to noise and hardware faults. This work introduces the Hyperdimensional Contextual Bandits (HD-CB): the first exploration of HDC to model and automate sequential decision-making Contextual Bandits (CB) problems. The proposed approach maps environmental states in a high-dimensional space and represents each action with dedicated hypervectors (HVs). At each iteration, these HVs are used to select the optimal action for the given context and are updated based on the received reward, replacing computationally expensive ridge regression procedures required by traditional linear CB algorithms with simple, highly parallel vector operations. We propose four HD-CB variants, demonstrating their flexibility in implementing different exploration strategies, as well as techniques to reduce memory overhead and the number of hyperparameters. Extensive simulations on synthetic datasets and a real-world benchmark reveal that HD-CB consistently achieves competitive or superior performance compared to traditional linear CB algorithms, while offering faster convergence time, lower computational complexity, improved scalability, and high parallelism.