Model selection-based estimation for generalized additive models using mixtures of g-priors: Towards systematization

TL;DR

This paper develops a Bayesian model selection framework for generalized additive models using mixtures of g-priors, extending traditional methods to non-Gaussian distributions via Laplace approximation, and demonstrates its superior performance through simulations.

Contribution

It introduces a systematic approach for model selection in generalized additive models with non-Gaussian responses using mixtures of g-priors, expanding beyond Gaussian regression.

Findings

Mixtures of g-priors outperform classical priors in nonparametric regression.

Model selection-based methods outperform other Bayesian approaches in simulations.

Optimal prior choices for knots improve model estimation accuracy.

Abstract

We explore the estimation of generalized additive models using basis expansion in conjunction with Bayesian model selection. Although Bayesian model selection is useful for regression splines, it has traditionally been applied mainly to Gaussian regression owing to the availability of a tractable marginal likelihood. We extend this method to handle an exponential family of distributions by using the Laplace approximation of the likelihood. Although this approach works well with any Gaussian prior distribution, consensus has not been reached on the best prior for nonparametric regression with basis expansions. Our investigation indicates that the classical unit information prior may not be ideal for nonparametric regression. Instead, we find that mixtures of g-priors are more effective. We evaluate various mixtures of g-priors to assess their performance in estimating generalized additive models. Additionally, we compare several priors for knots to determine the most effective strategy. Our simulation studies demonstrate that model selection-based approaches outperform other Bayesian methods.