HDXplore: Automated Blackbox Testing of Brain-Inspired Hyperdimensional Computing

TL;DR

HDXplore is a framework that systematically tests hyperdimensional computing models for robustness issues using blackbox differential testing, uncovering many incorrect behaviors and improving accuracy through retraining.

Contribution

This paper introduces HDXplore, the first automated blackbox testing framework specifically designed for hyperdimensional computing models, revealing their robustness weaknesses.

Findings

HDXplore finds thousands of incorrect behaviors in HDC models.

Retraining with corner cases improves HDC model accuracy by up to 9%.

The framework effectively exposes robustness issues without manual labeling.

Abstract

Inspired by the way human brain works, the emerging hyperdimensional computing (HDC) is getting more and more attention. HDC is an emerging computing scheme based on the working mechanism of brain that computes with deep and abstract patterns of neural activity instead of actual numbers. Compared with traditional ML algorithms such as DNN, HDC is more memory-centric, granting it advantages such as relatively smaller model size, less computation cost, and one-shot learning, making it a promising candidate in low-cost computing platforms. However, the robustness of HDC models have not been systematically studied. In this paper, we systematically expose the unexpected or incorrect behaviors of HDC models by developing HDXplore, a blackbox differential testing-based framework. We leverage multiple HDC models with similar functionality as cross-referencing oracles to avoid manual checking or labeling the original input. We also propose different perturbation mechanisms in HDXplore. HDXplore automatically finds thousands of incorrect corner case behaviors of the HDC model. We propose two retraining mechanisms and using the corner cases generated by HDXplore to retrain the HDC model, we can improve the model accuracy by up to 9%.