Do Quantum Neural Networks have Simplicity Bias?

TL;DR

This paper investigates the inductive bias and expressivity of quantum neural networks (QNNs), revealing a tradeoff where QNNs either lack bias or expressivity, limiting their advantage over classical deep neural networks.

Contribution

It provides a theoretical analysis of the bias-expressivity tradeoff in QNNs and demonstrates how restricting expressivity can induce inductive bias.

Findings

QNNs can exhibit simplicity bias under certain conditions

High expressivity in QNNs often results in poor inductive bias

QNNs studied do not outperform DNNs due to bias or expressivity limitations

Abstract

One hypothesis for the success of deep neural networks (DNNs) is that they are highly expressive, which enables them to be applied to many problems, and they have a strong inductive bias towards solutions that are simple, known as simplicity bias, which allows them to generalise well on unseen data because most real-world data is structured (i.e. simple). In this work, we explore the inductive bias and expressivity of quantum neural networks (QNNs), which gives us a way to compare their performance to those of DNNs. Our results show that it is possible to have simplicity bias with certain QNNs, but we prove that this type of QNN limits the expressivity of the QNN. We also show that it is possible to have QNNs with high expressivity, but they either have no inductive bias or a poor inductive bias and result in a worse generalisation performance compared to DNNs. We demonstrate that an artificial (restricted) inductive bias can be produced by intentionally restricting the expressivity of a QNN. Our results suggest a bias-expressivity tradeoff. Our conclusion is that the QNNs we studied can not generally offer an advantage over DNNs, because these QNNs either have a poor inductive bias or poor expressivity compared to DNNs.