Quantum Neuron Selection: Finding High Performing Subnetworks With Quantum Algorithms

TL;DR

This paper investigates how quantum algorithms can efficiently identify high-performing subnetworks within large neural networks, potentially revolutionizing neural network training by bypassing traditional gradient descent methods.

Contribution

It introduces quantum algorithms for local neuron selection that reduce complexity, making subnetwork identification more feasible on current quantum hardware.

Findings

Quantum algorithms can effectively identify high-performing subnetworks.

Proposed methods reduce entanglement complexity for quantum neuron selection.

Potential for quantum-assisted neural network pruning and training.

Abstract

Gradient descent methods have long been the de facto standard for training deep neural networks. Millions of training samples are fed into models with billions of parameters, which are slowly updated over hundreds of epochs. Recently, it's been shown that large, randomly initialized neural networks contain subnetworks that perform as well as fully trained models. This insight offers a promising avenue for training future neural networks by simply pruning weights from large, random models. However, this problem is combinatorically hard and classical algorithms are not efficient at finding the best subnetwork. In this paper, we explore how quantum algorithms could be formulated and applied to this neuron selection problem. We introduce several methods for local quantum neuron selection that reduce the entanglement complexity that large scale neuron selection would require, making this problem more tractable for current quantum hardware.