High-dimensional sparse classification using exponential weighting with empirical hinge loss

TL;DR

This paper introduces a novel high-dimensional binary classification method using exponential weighting with empirical hinge loss, achieving better prediction accuracy than logistic Lasso through theoretical guarantees and efficient Langevin Monte Carlo sampling.

Contribution

The paper proposes a new aggregation technique with sparsity priors and Langevin Monte Carlo for high-dimensional classification, showing improved performance over existing methods.

Findings

Favorable theoretical bounds on prediction error.

Superior performance compared to logistic Lasso in simulations.

Effective use of Langevin Monte Carlo for efficient sampling.

Abstract

In this study, we address the problem of high-dimensional binary classification. Our proposed solution involves employing an aggregation technique founded on exponential weights and empirical hinge loss. Through the employment of a suitable sparsity-inducing prior distribution, we demonstrate that our method yields favorable theoretical results on prediction error. The efficiency of our procedure is achieved through the utilization of Langevin Monte Carlo, a gradient-based sampling approach. To illustrate the effectiveness of our approach, we conduct comparisons with the logistic Lasso on simulated data and a real dataset. Our method frequently demonstrates superior performance compared to the logistic Lasso.