Enhanced Laplace Approximation

TL;DR

This paper introduces an enhanced Laplace approximation (ELA) that accurately estimates the maximum likelihood and variance for complex models with latent variables, improving upon traditional methods.

Contribution

The paper proposes the ELA method that yields true MLEs and consistent variance estimators for complex models, addressing limitations of existing Laplace approximations.

Findings

ELA provides accurate MLE and variance estimates for fixed parameters.

Numerical studies confirm ELA's effectiveness in complex models.

ELA offers a practical alternative to Bayesian procedures with different priors.

Abstract

The Laplace approximation (LA) has been proposed as a method for approximating the marginal likelihood of statistical models with latent variables. However, the approximate maximum likelihood estimators (MLEs) based on the LA are often biased for binary or spatial data, and the corresponding Hessian matrix underestimates the standard errors of these approximate MLEs. A higher-order approximation has been proposed; however, it cannot be applied to complicated models such as correlated random effects models and does not provide consistent variance estimators. In this paper, we propose an enhanced LA (ELA) that provides the true MLE and its consistent variance estimator. We study its relationship to the variational Bayes method. We also introduce a new restricted maximum likelihood estimator (REMLE) for estimating dispersion parameters. The results of numerical studies show that the ELA provides a satisfactory MLE and REMLE, as well as their variance estimators for fixed parameters. The MLE and REMLE can be viewed as posterior mode and marginal posterior mode under flat priors, respectively. Some comparisons are also made with Bayesian procedures under different priors.