Bayesian Projection Pursuit Regression

TL;DR

This paper introduces a Bayesian approach to projection pursuit regression, enabling uncertainty quantification and automatic model complexity selection, and demonstrates its effectiveness through extensive simulations and real data applications.

Contribution

It is the first Bayesian formulation of PPR, incorporating reversible jump MCMC to determine the number of ridge functions, enhancing model flexibility and interpretability.

Findings

Competitive predictive performance in simulations and real datasets

Effective uncertainty quantification for model parameters

Automatic selection of the number of ridge functions

Abstract

In projection pursuit regression (PPR), an unknown response function is approximated by the sum of M "ridge functions," which are flexible functions of one-dimensional projections of a multivariate input space. Traditionally, optimization routines are used to estimate the projection directions and ridge functions via a sequential algorithm, and M is typically chosen via cross-validation. We introduce the first Bayesian version of PPR, which has the benefit of accurate uncertainty quantification. To learn the projection directions and ridge functions, we apply novel adaptations of methods used for the single ridge function case (M=1), called the Single Index Model, for which Bayesian implementations do exist; then use reversible jump MCMC to learn the number of ridge functions M. We evaluate the predictive ability of our model in 20 simulation scenarios and for 23 real datasets, in a bake-off against an array of state-of-the-art regression methods. Its effective performance indicates that Bayesian Projection Pursuit Regression is a valuable addition to the existing regression toolbox.