Adaptive Monte Carlo Maximum Likelihood

TL;DR

This paper introduces adaptive Monte Carlo algorithms for maximum likelihood estimation in models with intractable constants, improving efficiency and reducing weight degeneracy through resampling and MCMC techniques.

Contribution

It develops new adaptive importance sampling algorithms with resampling and MCMC, and analyzes their asymptotic properties using martingale limit theorems.

Findings

Adaptive algorithms outperform non-adaptive methods in simulations

Resampling reduces importance weight degeneracy

Theoretical analysis confirms convergence properties

Abstract

We consider Monte Carlo approximations to the maximum likelihood estimator in models with intractable norming constants. This paper deals with adaptive Monte Carlo algorithms, which adjust control parameters in the course of simulation. We examine asymptotics of adaptive importance sampling and a new algorithm, which uses resampling and MCMC. This algorithm is designed to reduce problems with degeneracy of importance weights. Our analysis is based on martingale limit theorems. We also describe how adaptive maximization algorithms of Newton-Raphson type can be combined with the resampling techniques. The paper includes results of a small scale simulation study in which we compare the performance of adaptive and non-adaptive Monte Carlo maximum likelihood algorithms.