Approximate Inference for Observation Driven Time Series Models with Intractable Likelihoods

TL;DR

This paper develops an approximate Bayesian inference method for observation driven time series models with intractable likelihoods, introducing a new MCMC kernel that improves computational efficiency and asymptotic convergence.

Contribution

It proposes a novel ABC-based approximation and an improved MCMC algorithm for inference in models with intractable likelihoods, ensuring asymptotic consistency.

Findings

The ABC posterior's MAP estimator converges to a biased limit as data length increases.

A noisy ABC MAP estimator converges to the true parameter asymptotically.

The new MCMC kernel has an expected cost of O(n^2) per iteration, improving efficiency.

Abstract

In the following article we consider approximate Bayesian parameter inference for observation driven time series models. Such statistical models appear in a wide variety of applications, including econometrics and applied mathematics. This article considers the scenario where the likelihood function cannot be evaluated point-wise; in such cases, one cannot perform exact statistical inference, including parameter estimation, which often requires advanced computational algorithms, such as Markov chain Monte Carlo (MCMC). We introduce a new approximation based upon approximate Bayesian computation (ABC). Under some conditions, we show that as $n\rightarrow\infty$, with $n$ the length of the time series, the ABC posterior has, almost surely, a maximum \emph{a posteriori} (MAP) estimator of the parameters which is different from the true parameter. However, a noisy ABC MAP, which perturbs the original data, asymptotically converges to the true parameter, almost surely. In order to draw statistical inference, for the ABC approximation adopted, standard MCMC algorithms can have acceptance probabilities that fall at an exponential rate in $n$ and slightly more advanced algorithms can mix poorly. We develop a new and improved MCMC kernel, which is based upon an exact approximation of a marginal algorithm, whose cost per-iteration is random but the expected cost, for good performance, is shown to be $\mathcal{O}(n^2)$ per-iteration.