Physics-Inspired Binary Neural Networks: Interpretable Compression with Theoretical Guarantees

TL;DR

This paper introduces PIBiNN, a physics-inspired binary neural network that leverages problem-specific sparsity and one-bit quantization to achieve highly compressed models with preserved geometry and improved generalization.

Contribution

The work presents a novel binary neural network architecture that integrates physics-based sparsity and one-bit quantization, enabling ultra-compression without ad-hoc pruning.

Findings

Achieves compression rates below one bit per weight.

Outperforms ternary and channel-wise quantization in memory efficiency.

Maintains essential operator geometry and improves generalization.

Abstract

Why rely on dense neural networks and then blindly sparsify them when prior knowledge about the problem structure is already available? Many inverse problems admit algorithm-unrolled networks that naturally encode physics and sparsity. In this work, we propose a Physics-Inspired Binary Neural Network (PIBiNN) that combines two key components: (i) data-driven one-bit quantization with a single global scale, and (ii) problem-driven sparsity predefined by physics and requiring no updates during training. This design yields compression rates below one bit per weight by exploiting structural zeros, while preserving essential operator geometry. Unlike ternary or pruning-based schemes, our approach avoids ad-hoc sparsification, reduces metadata overhead, and aligns directly with the underlying task. Experiments suggest that PIBiNN achieves advantages in both memory efficiency and generalization compared to competitive baselines such as ternary and channel-wise quantization.