Latent Gaussian Model Boosting

TL;DR

This paper introduces a novel method combining latent Gaussian models with boosting to improve prediction accuracy, address dependence modeling, and handle high-cardinality categorical variables more effectively.

Contribution

It presents a new approach that integrates boosting with latent Gaussian models, overcoming limitations of existing methods and enhancing predictive performance.

Findings

Improved prediction accuracy on simulated data

Enhanced modeling of dependence among samples

Effective handling of high-cardinality categorical variables

Abstract

Latent Gaussian models and boosting are widely used techniques in statistics and machine learning. Tree-boosting shows excellent prediction accuracy on many data sets, but potential drawbacks are that it assumes conditional independence of samples, produces discontinuous predictions for, e.g., spatial data, and it can have difficulty with high-cardinality categorical variables. Latent Gaussian models, such as Gaussian process and grouped random effects models, are flexible prior models which explicitly model dependence among samples and which allow for efficient learning of predictor functions and for making probabilistic predictions. However, existing latent Gaussian models usually assume either a zero or a linear prior mean function which can be an unrealistic assumption. This article introduces a novel approach that combines boosting and latent Gaussian models to remedy the above-mentioned drawbacks and to leverage the advantages of both techniques. We obtain increased prediction accuracy compared to existing approaches in both simulated and real-world data experiments.