Variational Inference for Quantum HyperNetworks

TL;DR

This paper introduces a quantum-inspired Bayesian inference framework for training Binary Neural Networks using variational quantum algorithms, leading to improved trainability and generalization in large-scale systems.

Contribution

It connects quantum hypernetworks with Bayesian inference by deriving ELBO and MMD-based surrogate, enhancing BiNN training with quantum techniques.

Findings

Outperforms standard MLE in experiments

Improves trainability of Binary Neural Networks

Enhances generalization capabilities

Abstract

Binary Neural Networks (BiNNs), which employ single-bit precision weights, have emerged as a promising solution to reduce memory usage and power consumption while maintaining competitive performance in large-scale systems. However, training BiNNs remains a significant challenge due to the limitations of conventional training algorithms. Quantum HyperNetworks offer a novel paradigm for enhancing the optimization of BiNN by leveraging quantum computing. Specifically, a Variational Quantum Algorithm is employed to generate binary weights through quantum circuit measurements, while key quantum phenomena such as superposition and entanglement facilitate the exploration of a broader solution space. In this work, we establish a connection between this approach and Bayesian inference by deriving the Evidence Lower Bound (ELBO), when direct access to the output distribution is available (i.e., in simulations), and introducing a surrogate ELBO based on the Maximum Mean Discrepancy (MMD) metric for scenarios involving implicit distributions, as commonly encountered in practice. Our experimental results demonstrate that the proposed methods outperform standard Maximum Likelihood Estimation (MLE), improving trainability and generalization.