Variational Inference for Variable Selection in Scalar-on-Function Regression

TL;DR

This paper introduces a variational inference method for variable selection in scalar-on-function regression, effectively identifying relevant covariates and improving prediction accuracy in models with functional and scalar predictors.

Contribution

It develops a novel variational EM algorithm for variable selection in scalar-on-function regression, outperforming existing methods in accuracy and sparsity.

Findings

Accurately identifies relevant covariates in simulations.

Achieves better balance between fit and sparsity than alternatives.

Demonstrates practical utility in real-world spectral and weather data.

Abstract

In practical regression applications, multiple covariates are often measured, but not all may be associated with the response variable. Identifying and including only the relevant covariates in the model is crucial for improving prediction accuracy. In this work, we develop a variational inference approach for estimation and variable selection in scalar-on-function regression, involving only functional covariates, and in partially functional regression models that also include scalar covariates. Specifically, we develop a variational expectation-maximization (VEM) algorithm, with a variational Bayes procedure implemented in the E-step to obtain approximate marginal posterior distributions for most model parameters, except for the regularization parameters, which are updated in the M-step. Our method accurately identifies relevant covariates while maintaining strong predictive performance, as demonstrated through extensive simulation studies across diverse scenarios. Compared with alternative approaches, including BGLSS (Bayesian Group Lasso with Spike-and-Slab priors), grLASSO (group Least Absolute Shrinkage and Selection Operator), grMCP (group Minimax Concave Penalty), and grSCAD (group Smoothly Clipped Absolute Deviation), our approach achieves a superior balance between goodness-of-fit and sparsity in most scenarios. We further illustrate its practical utility through real-data applications involving spectral analysis of sugar samples and weather measurements from Japan.