The inefficiency of re-weighted sampling and the curse of system size in high order path integration

TL;DR

This paper analyzes the limitations of re-weighted sampling methods for high-order path integrals, revealing that their efficiency diminishes exponentially with system size due to increasing statistical uncertainty.

Contribution

It provides a general theoretical analysis of re-weighted sampling inefficiencies and demonstrates their failure in large systems, especially for high-order path integral techniques.

Findings

Re-weighted sampling accuracy deteriorates exponentially with system size.

High-order path integral methods based on re-weighting are impractical for large systems.

Statistical uncertainty grows exponentially, limiting the feasibility of re-weighted approaches.

Abstract

Computing averages over a target probability density by statistical re-weighting of a set of samples with a different distribution is a strategy which is commonly adopted in fields as diverse as atomistic simulation and finance. Here we present a very general analysis of the accuracy and efficiency of this approach, highlighting some of its weaknesses. We then give an example of how our results can be used, specifically to assess the feasibility of high-order path integral methods. We demonstrate that the most promising of these techniques -- which is based on re-weighted sampling -- is bound to fail as the size of the system is increased, because of the exponential growth of the statistical uncertainty in the re-weighted average.