Negative association, ordering and convergence of resampling methods

TL;DR

This paper analyzes convergence properties of various resampling methods in particle algorithms, introducing a new algorithm with guaranteed convergence and demonstrating how ordering particles affects convergence rates.

Contribution

It introduces a new resampling algorithm based on stochastic rounding that guarantees convergence regardless of input order and proves faster convergence when particles are ordered via the Hilbert curve.

Findings

The new stochastic rounding resampling method converges regardless of input order.

Ordering particles with the Hilbert curve improves convergence rate to ${\ extstyle\mathcal{O}}(N^{-(1+1/d)})$.

Systematic resampling's convergence can depend on input order, unlike the new method.

Abstract

We study convergence and convergence rates for resampling schemes. Our first main result is a general consistency theorem based on the notion of negative association, which is applied to establish the almost-sure weak convergence of measures output from Kitagawa's (1996) stratified resampling method. Carpenter et al's (1999) systematic resampling method is similar in structure but can fail to converge depending on the order of the input samples. We introduce a new resampling algorithm based on a stochastic rounding technique of Srinivasan (2001), which shares some attractive properties of systematic resampling, but which exhibits negative association and therefore converges irrespective of the order of the input samples. We confirm a conjecture made by Kitagawa (1996) that ordering input samples by their states in $\mathbb{R}$ yields a faster rate of convergence; we establish that when particles are ordered using the Hilbert curve in $\mathbb{R}^d$, the variance of the resampling error is ${\scriptscriptstyle\mathcal{O}}(N^{-(1+1/d)})$ under mild conditions, where $N$ is the number of particles. We use these results to establish asymptotic properties of particle algorithms based on resampling schemes that differ from multinomial resampling.