On the Properties of Simulation-based Estimators in High Dimensions

TL;DR

This paper develops a general framework for simulation-based estimators in high-dimensional settings, ensuring properties like unbiasedness, consistency, and asymptotic normality, and introduces an efficient iterative bootstrap algorithm.

Contribution

It extends simulation-based estimation methods to high-dimensional models, including discrete and large-parameter models, with proven convergence and statistical properties.

Findings

The Iterative Bootstrap algorithm converges efficiently.

Simulation-based estimators are unbiased, consistent, and asymptotically normal in high dimensions.

Application to logistic, negative binomial, and lasso regression models demonstrates practical effectiveness.

Abstract

Considering the increasing size of available data, the need for statistical methods that control the finite sample bias is growing. This is mainly due to the frequent settings where the number of variables is large and allowed to increase with the sample size bringing standard inferential procedures to incur significant loss in terms of performance. Moreover, the complexity of statistical models is also increasing thereby entailing important computational challenges in constructing new estimators or in implementing classical ones. A trade-off between numerical complexity and statistical properties is often accepted. However, numerically efficient estimators that are altogether unbiased, consistent and asymptotically normal in high dimensional problems would generally be ideal. In this paper, we set a general framework from which such estimators can easily be derived for wide classes of models. This framework is based on the concepts that underlie simulation-based estimation methods such as indirect inference. The approach allows various extensions compared to previous results as it is adapted to possibly inconsistent estimators and is applicable to discrete models and/or models with a large number of parameters. We consider an algorithm, namely the Iterative Bootstrap (IB), to efficiently compute simulation-based estimators by showing its convergence properties. Within this framework we also prove the properties of simulation-based estimators, more specifically the unbiasedness, consistency and asymptotic normality when the number of parameters is allowed to increase with the sample size. Therefore, an important implication of the proposed approach is that it allows to obtain unbiased estimators in finite samples. Finally, we study this approach when applied to three common models, namely logistic regression, negative binomial regression and lasso regression.