Regularization and Variable Selection with Copula Prior

TL;DR

This paper introduces the copula prior as a flexible regularization method that generalizes existing techniques like lasso and elastic net, demonstrating improved performance and grouping effects in regression and classification tasks.

Contribution

It proposes the copula prior framework, including specific cases like Gauss and t-copula priors, and develops a resampling-based optimization for large datasets.

Findings

Copula prior often outperforms lasso and elastic net in predictive accuracy.

Encourages grouping of strongly correlated predictors.

Effective in regression, classification, and large time-series data.

Abstract

In this work, we show that under specific choices of the copula, the lasso, elastic net, and $g$-prior are particular cases of `copula prior,' for regularization and variable selection method. We present `lasso with Gauss copula prior' and `lasso with t-copula prior.' The simulation study and real-world data for regression, classification, and large time-series data show that the `copula prior' often outperforms the lasso and elastic net while having a comparable sparsity of representation. Also, the copula prior encourages a grouping effect. The strongly correlated predictors tend to be in or out of the model collectively under the copula prior. The `copula prior' is a generic method, which can be used to define the new prior distribution. The application of copulas in modeling prior distribution for Bayesian methodology has not been explored much. We present the resampling-based optimization procedure to handle big data with copula prior.