G-Net: A Provably Easy Construction of High-Accuracy Random Binary Neural Networks

TL;DR

This paper introduces G-Net, a new randomized binary neural network construction method inspired by hyperdimensional computing, achieving high accuracy and robustness with theoretical guarantees, and outperforming prior models on benchmarks like CIFAR-10.

Contribution

The paper presents G-Net, a novel family of floating-point and binary neural networks with provable accuracy guarantees, bridging neural networks and hyperdimensional computing.

Findings

Binary models match CNN accuracies

Achieve nearly 30% higher accuracy on CIFAR-10 than prior HDC models

Theoretical guarantees due to concentration of measure

Abstract

We propose a novel randomized algorithm for constructing binary neural networks with tunable accuracy. This approach is motivated by hyperdimensional computing (HDC), which is a brain-inspired paradigm that leverages high-dimensional vector representations, offering efficient hardware implementation and robustness to model corruptions. Unlike traditional low-precision methods that use quantization, we consider binary embeddings of data as points in the hypercube equipped with the Hamming distance. We propose a novel family of floating-point neural networks, G-Nets, which are general enough to mimic standard network layers. Each floating-point G-Net has a randomized binary embedding, an embedded hyperdimensional (EHD) G-Net, that retains the accuracy of its floating-point counterparts, with theoretical guarantees, due to the concentration of measure. Empirically, our binary models match convolutional neural network accuracies and outperform prior HDC models by large margins, for example, we achieve almost 30\% higher accuracy on CIFAR-10 compared to prior HDC models. G-Nets are a theoretically justified bridge between neural networks and randomized binary neural networks, opening a new direction for constructing robust binary/quantized deep learning models. Our implementation is available at https://github.com/GNet2025/GNet.