Estimation and Inference by Stochastic Optimization

TL;DR

This paper introduces two stochastic optimization methods, rNR and rqN, that efficiently combine estimation and inference by resampling, reducing computational costs in complex models.

Contribution

It presents novel resampled Newton-Raphson and quasi-Newton algorithms that enable fast inference within a single optimization run for complex models.

Findings

Methods perform well on large-scale problems

Algorithms provide consistent estimates and confidence intervals

Compared favorably with traditional frequentist and Bayesian approaches

Abstract

In non-linear estimations, it is common to assess sampling uncertainty by bootstrap inference. For complex models, this can be computationally intensive. This paper combines optimization with resampling: turning stochastic optimization into a fast resampling device. Two methods are introduced: a resampled Newton-Raphson (rNR) and a resampled quasi-Newton (rqN) algorithm. Both produce draws that can be used to compute consistent estimates, confidence intervals, and standard errors in a single run. The draws are generated by a gradient and Hessian (or an approximation) computed from batches of data that are resampled at each iteration. The proposed methods transition quickly from optimization to resampling when the objective is smooth and strictly convex. Simulated and empirical applications illustrate the properties of the methods on large scale and computationally intensive problems. Comparisons with frequentist and Bayesian methods highlight the features of the algorithms.