Bayesian Smoothing and Feature Selection Using variational Automatic Relevance Determination

TL;DR

This paper presents VARD, a variational method for sparse additive regression that assesses feature relevance and smoothness independently, with an efficient algorithm and demonstrated superior performance on simulated and real data.

Contribution

Introduces VARD, a novel variational approach for automatic relevance determination in high-dimensional additive models, enabling independent feature smoothness assessment and precise relevance determination.

Findings

VARD outperforms existing variable selection methods in experiments.

Efficient coordinate descent algorithm for VARD.

Effective in high-dimensional settings with simulated and real data.

Abstract

This study introduces Variational Automatic Relevance Determination (VARD), a novel approach tailored for fitting sparse additive regression models in high-dimensional settings. VARD distinguishes itself by its ability to independently assess the smoothness of each feature while enabling precise determination of whether a feature's contribution to the response is zero, linear, or nonlinear. Further, an efficient coordinate descent algorithm is introduced to implement VARD. Empirical evaluations on simulated and real-world data underscore VARD's superiority over alternative variable selection methods for additive models.