Replicated Vector Approximate Message Passing For Resampling Problem

TL;DR

This paper introduces a fast, approximate inference method combining the replica method and vector approximate message passing to efficiently perform resampling in statistical inference, reducing computational costs.

Contribution

It presents a novel computationally efficient approach for resampling that avoids repeated estimation, applicable to stability selection in variable selection problems.

Findings

Fast convergence demonstrated in synthetic and real datasets

High approximation accuracy achieved

Reduces computational time compared to traditional methods

Abstract

Resampling techniques are widely used in statistical inference and ensemble learning, in which estimators' statistical properties are essential. However, existing methods are computationally demanding, because repetitions of estimation/learning via numerical optimization/integral for each resampled data are required. In this study, we introduce a computationally efficient method to resolve such problem: replicated vector approximate message passing. This is based on a combination of the replica method of statistical physics and an accurate approximate inference algorithm, namely the vector approximate message passing of information theory. The method provides tractable densities without repeating estimation/learning, and the densities approximately offer an arbitrary degree of the estimators' moment in practical time. In the experiment, we apply the proposed method to the stability selection method, which is commonly used in variable selection problems. The numerical results show its fast convergence and high approximation accuracy for problems involving both synthetic and real-world datasets.