Empirical Likelihood Based Bayesian Variable Selection

TL;DR

This paper introduces a Bayesian empirical likelihood approach for variable selection that is theoretically consistent and computationally efficient, applicable with various priors including LASSO, SCAD, ridge, and elastic net.

Contribution

It develops a novel Bayesian empirical likelihood method with proven consistency and introduces an approximation approach to improve MCMC convergence.

Findings

Bayesian empirical likelihood achieves posterior and variable selection consistency.

The proposed approximation method enhances MCMC convergence speed.

Empirical and real data analyses demonstrate the method's effectiveness.

Abstract

Empirical likelihood is a popular nonparametric statistical tool that does not require any distributional assumptions. In this paper, we explore the possibility of conducting variable selection via Bayesian empirical likelihood. We show theoretically that when the prior distribution satisfies certain mild conditions, the corresponding Bayesian empirical likelihood estimators are posteriorly consistent and variable selection consistent. As special cases, we show the prior of Bayesian empirical likelihood LASSO and SCAD satisfies such conditions and thus can identify the non-zero elements of the parameters with probability tending to 1. In addition, it is easy to verify that those conditions are met for other widely used priors such as ridge, elastic net and adaptive LASSO. Empirical likelihood depends on a parameter that needs to be obtained by numerically solving a non-linear equation. Thus, there exists no conjugate prior for the posterior distribution, which causes the slow convergence of the MCMC sampling algorithm in some cases. To solve this problem, we propose a novel approach, which uses an approximation distribution as the proposal. The computational results demonstrate quick convergence for the examples used in the paper. We use both simulation and real data analyses to illustrate the advantages of the proposed methods.