Asymptotic accuracy of the saddlepoint approximation for maximum likelihood estimation

TL;DR

This paper establishes that the saddlepoint approximation for maximum likelihood estimation has an asymptotically negligible error compared to the statistical uncertainty, with errors decreasing at rates of O(1/n^2) or O(1/n).

Contribution

It proves the asymptotic accuracy of the saddlepoint MLE and introduces a new, simpler saddlepoint approximation with similar error bounds.

Findings

Saddlepoint MLE error is asymptotically negligible compared to inferential uncertainty.

Error bounds are established as O(1/n^2), O(1/n^{3/2}), or O(1/n) for different parameters.

A new simpler saddlepoint approximation with comparable error bounds is proposed.

Abstract

The saddlepoint approximation gives an approximation to the density of a random variable in terms of its moment generating function. When the underlying random variable is itself the sum of $n$ unobserved i.i.d. terms, the basic classical result is that the relative error in the density is of order $1/n$. If instead the approximation is interpreted as a likelihood and maximised as a function of model parameters, the result is an approximation to the maximum likelihood estimate (MLE) that can be much faster to compute than the true MLE. This paper proves the analogous basic result for the approximation error between the saddlepoint MLE and the true MLE: subject to certain explicit identifiability conditions, the error has asymptotic size $O(1/n^2)$ for some parameters, and $O(1/n^{3/2})$ or $O(1/n)$ for others. In all three cases, the approximation errors are asymptotically negligible compared to the inferential uncertainty. The proof is based on a factorisation of the saddlepoint likelihood into an exact and approximate term, along with an analysis of the approximation error in the gradient of the log-likelihood. This factorisation also gives insight into alternatives to the saddlepoint approximation, including a new and simpler saddlepoint approximation, for which we derive analogous error bounds. As a corollary of our results, we also obtain the asymptotic size of the MLE error approximation when the saddlepoint approximation is replaced by the normal approximation.