High-dimensional prediction for count response via sparse exponential weights

TL;DR

This paper introduces a new probabilistic framework for high-dimensional count data prediction, combining a sparse prior and exponential weights, with theoretical guarantees and superior performance over Lasso.

Contribution

It develops a pseudo-Bayesian method with PAC-Bayesian guarantees for count data prediction, a novel risk measure, and efficient implementation via Langevin Monte Carlo.

Findings

Non-asymptotic oracle inequalities with optimal rates

Strong empirical performance compared to Lasso

Theoretical prediction guarantees for high-dimensional count data

Abstract

Count data is prevalent in various fields like ecology, medical research, and genomics. In high-dimensional settings, where the number of features exceeds the sample size, feature selection becomes essential. While frequentist methods like Lasso have advanced in handling high-dimensional count data, Bayesian approaches remain under-explored with no theoretical results on prediction performance. This paper introduces a novel probabilistic machine learning framework for high-dimensional count data prediction. We propose a pseudo-Bayesian method that integrates a scaled Student prior to promote sparsity and uses an exponential weight aggregation procedure. A key contribution is a novel risk measure tailored to count data prediction, with theoretical guarantees for prediction risk using PAC-Bayesian bounds. Our results include non-asymptotic oracle inequalities, demonstrating rate-optimal prediction error without prior knowledge of sparsity. We implement this approach efficiently using Langevin Monte Carlo method. Simulations and a real data application highlight the strong performance of our method compared to the Lasso in various settings.