Projection Predictive Inference for Generalized Linear and Additive Multilevel Models

TL;DR

This paper extends projection predictive inference to generalized linear and additive multilevel models, enabling effective variable and structure selection that reduces model complexity while maintaining predictive accuracy.

Contribution

It introduces a novel extension of projection predictive inference to GLMMs and GAMMs, enhancing variable selection in complex multilevel models.

Findings

Significantly reduces model complexity needed for accurate predictions.

Achieves good frequency properties in model selection.

Demonstrates superior performance over existing methods.

Abstract

Projection predictive inference is a decision theoretic Bayesian approach that decouples model estimation from decision making. Given a reference model previously built including all variables present in the data, projection predictive inference projects its posterior onto a constrained space of a subset of variables. Variable selection is then performed by sequentially adding relevant variables until predictive performance is satisfactory. Previously, projection predictive inference has been demonstrated only for generalized linear models (GLMs) and Gaussian processes (GPs) where it showed superior performance to competing variable selection procedures. In this work, we extend projection predictive inference to support variable and structure selection for generalized linear multilevel models (GLMMs) and generalized additive multilevel models (GAMMs). Our simulative and real-word experiments demonstrate that our method can drastically reduce the model complexity required to reach reference predictive performance and achieve good frequency properties.