Overlapped groupings for quantum energy estimation: Maximal variance reduction and deterministic algorithms for reducing variance

TL;DR

This paper proves that overlapped grouping strategies can maximally reduce variance in quantum energy estimation, introduces a new repacking algorithm, and demonstrates significant variance reduction in large-scale simulations.

Contribution

It provides a theoretical proof of maximal variance reduction via overlapped grouping, introduces a repacking algorithm, and validates its effectiveness through large-scale numerical simulations.

Findings

Overlapped grouping can linearly reduce variance with the number of Hamiltonian terms.

The repacking algorithm iteratively decreases variance.

Numerical results show increased variance reduction at large scales.

Abstract

Grouping-based measurement strategies are widely used to reduce measurement complexity in near-term quantum algorithms. While these schemes have typically produced disjoint groups, recently this has been relaxed in what is known as overlapped grouping or coefficient splitting where operators may appear in more than one compatible group. In recent work, it has been numerically shown that this strategy can reduce the variance of energy estimates on small benchmark problems, motivating both the application and further analysis of the method. Here we prove that overlapped grouping for energy estimation can lead to a maximal variance reduction that is linear in the number of Hamiltonian terms. We introduce a new algorithm which we call repacking to transform existing groups into overlapped groups, and we show this repacking procedure iteratively reduces variance under mild assumptions. We also perform numerical simulations with Hamiltonians up to $44$ qubits and $575 \cdot 10^{3}$ terms, assessing overlapped grouping at scale on problems of practical importance. Our numerics show that the variance reduction relative to state-of-the-art (disjoint) grouping increases linearly with the problem size, suggesting that overlapped grouping methods can be a powerful strategy for quantum energy estimation at the scale of Megaquop computers and beyond.