Higher Order Derivatives of Quantum Neural Networks with Barren Plateaus

TL;DR

This paper demonstrates that higher-order derivatives, including the Hessian, are exponentially suppressed in quantum neural networks experiencing barren plateaus, indicating that such derivatives cannot effectively help escape these optimization challenges.

Contribution

The authors derive novel formulas for evaluating high-order derivatives on quantum hardware and show that these derivatives are exponentially suppressed in barren plateau regions.

Findings

Hessian elements are exponentially suppressed in BPs

Higher order derivatives are also exponentially suppressed

Hessian-based methods do not overcome BPs

Abstract

Quantum neural networks (QNNs) offer a powerful paradigm for programming near-term quantum computers and have the potential to speedup applications ranging from data science to chemistry to materials science. However, a possible obstacle to realizing that speedup is the Barren Plateau (BP) phenomenon, whereby the gradient vanishes exponentially in the system size $n$ for certain QNN architectures. The question of whether high-order derivative information such as the Hessian could help escape a BP was recently posed in the literature. Here we show that the elements of the Hessian are exponentially suppressed in a BP, so estimating the Hessian in this situation would require a precision that scales exponentially with $n$. Hence, Hessian-based approaches do not circumvent the exponential scaling associated with BPs. We also show the exponential suppression of higher order derivatives. Hence, BPs will impact optimization strategies that go beyond (first-order) gradient descent. In deriving our results, we prove novel, general formulas that can be used to analytically evaluate any high-order partial derivative on quantum hardware. These formulas will likely have independent interest and use for training quantum neural networks (outside of the context of BPs).