Latent space projection predictive inference

TL;DR

This paper introduces a unified framework for variable selection in complex, non-exponential family models using latent space projection predictive inference, improving efficiency and accuracy while respecting original model structures.

Contribution

It extends projection predictive inference to non-exponential family models, enabling efficient variable and structure selection with full uncertainty quantification.

Findings

Successfully recovers relevant model terms

Selects fewer variables than competing methods

Demonstrates superior performance on realistic datasets

Abstract

Given a reference model that includes all the available variables, projection predictive inference replaces its posterior with a constrained projection including only a subset of all variables. We extend projection predictive inference to enable computationally efficient variable and structure selection in models outside the exponential family. By adopting a latent space projection predictive perspective we are able to: 1) propose a unified and general framework to do variable selection in complex models while fully honouring the original model structure, 2) properly identify relevant structure and retain posterior uncertainties from the original model, and 3) provide an improved approach also for non-Gaussian models in the exponential family. We demonstrate the superior performance of our approach by thoroughly testing and comparing it against popular variable selection approaches in a wide range of settings, including realistic data sets. Our results show that our approach successfully recovers relevant terms and model structure in complex models, selecting less variables than competing approaches for realistic datasets.