Bayesian inference under small sample size -- A noninformative prior approach

TL;DR

This paper introduces a Bayesian inference approach for small sample sizes using a broad class of noninformative priors, demonstrating improved prediction performance over traditional methods through numerical examples.

Contribution

It proposes a general noninformative prior ($ ightarrow 1/\sigma^{q}$) for small sample Bayesian inference, linking it to classical priors and enabling analytical posterior evaluation.

Findings

Jeffreys' prior yields the best predictor performance.

Noninformative Bayesian estimator outperforms least squares in small samples.

Laplace approximation effectively evaluates global likelihood.

Abstract

A Bayesian inference method for problems with small samples and sparse data is presented in this paper. A general type of prior ($\propto 1/\sigma^{q}$) is proposed to formulate the Bayesian posterior for inference problems under small sample size. It is shown that this type of prior can represents a broad range of priors such as classical noninformative priors and asymptotically locally invariant priors. It is further shown in this study that such priors can be derived as the limiting states of Normal-Inverse-Gamma conjugate priors, allowing for analytical evaluations of Bayesian posteriors and predictors. The performance of different noninformative priors under small sample size is compared using the global likelihood. The method of Laplace approximation is employed to evaluate the global likelihood. A numerical linear regression problem and a realistic fatigue reliability problem are used to demonstrate the method and identify the optimal noninformative prior. Results indicate the predictor using Jeffreys' prior outperforms others. The advantage of the noninformative Bayesian estimator over the regular least square estimator under small sample size is shown.