Estimating an Inverse Gamma distribution

TL;DR

This paper introduces five novel algorithms for estimating the parameters of the Inverse Gamma distribution, including the first conjugate prior for the shape parameter, and compares their performance using synthetic data.

Contribution

The paper presents new estimation algorithms for Inverse Gamma parameters, including the first conjugate prior and fast likelihood-based methods, enabling broader Bayesian modeling.

Findings

The algorithms are computationally compared using synthetic data.

The conjugate prior allows analytical Bayesian inference.

Likelihood approximation methods are very fast.

Abstract

In this paper we introduce five different algorithms based on method of moments, maximum likelihood and full Bayesian estimation for learning the parameters of the Inverse Gamma distribution. We also provide an expression for the KL divergence for Inverse Gamma distributions which allows us to quantify the estimation accuracy of each of the algorithms. All the presented algorithms are novel. The most relevant novelties include the first conjugate prior for the Inverse Gamma shape parameter which allows analytical Bayesian inference, and two very fast algorithms, a maximum likelihood and a Bayesian one, both based on likelihood approximation. In order to compute expectations under the proposed distributions we use the Laplace approximation. The introduction of these novel Bayesian estimators opens the possibility of including Inverse Gamma distributions into more complex Bayesian structures, e.g. variational Bayesian mixture models. The algorithms introduced in this paper are computationally compared using synthetic data and interesting relationships between the maximum likelihood and the Bayesian approaches are derived.