An Introduction to Inductive Statistical Inference: from Parameter Estimation to Decision-Making

TL;DR

This paper introduces Bayesian inductive inference, focusing on MCMC methods, model assessment, and decision-making, with practical examples and code implementations for applied statisticians and researchers.

Contribution

It provides a comprehensive overview of Bayesian inference techniques, including MCMC sampling, model comparison, and decision-making, with practical guidance and code in Stan and R.

Findings

Effective MCMC sampling for complex models

Model comparison using WAIC, LOOIC, and Bayes factors

Application of Bayesian methods to decision-making scenarios

Abstract

These lecture notes aim at a post-Bachelor audience with a background at an introductory level in Applied Mathematics and Applied Statistics. They discuss the logic and methodology of the Bayes-Laplace approach to inductive statistical inference that places common sense and the guiding lines of the scientific method at the heart of systematic analyses of quantitative-empirical data. Following an exposition of exactly solvable cases of single- and two-parameter estimation problems, the main focus is laid on Markov Chain Monte Carlo (MCMC) simulations on the basis of Hamiltonian Monte Carlo sampling of posterior joint probability distributions for regression parameters occurring in generalised linear models for a univariate outcome variable. The modelling of fixed effects as well as of correlated varying effects via multi-level models in non-centred parametrisation is considered. The simulation of posterior predictive distributions is outlined. The assessment of a model's relative out-of-sample posterior predictive accuracy with information entropy-based criteria WAIC and LOOIC and model comparison with Bayes factors are addressed. A brief discussion on the description of the generation of stationary time series data by means of autoregressive models is contained. Concluding, a conceptual link to the behavioural subjective expected utility representation of a single decision-maker's choice behaviour in static one-shot decision problems is established. Vectorised codes for MCMC simulations of multi-dimensional posterior joint probability distributions with the Stan probabilistic programming language implemented in the statistical software R are provided. The lecture notes are fully hyperlinked. They direct the reader to original scientific research papers, online resources on inductive statistical inference, and to pertinent biographical information.