Dimension-free PAC-Bayesian bounds for matrices, vectors, and linear least squares regression

TL;DR

This paper develops dimension-free PAC-Bayesian bounds applicable to heavy-tailed data, enhancing mean and Gram matrix estimation in high-dimensional linear regression.

Contribution

It introduces new PAC-Bayesian bounds that are dimension-free and robust to heavy tails, with specific focus on Gram matrix estimation.

Findings

Bounds are valid under weak polynomial moment assumptions.

Effective estimation methods for high-dimensional Gram matrices.

Applicable to heavy-tailed distributions in linear regression.

Abstract

This paper is focused on dimension-free PAC-Bayesian bounds, under weak polynomial moment assumptions, allowing for heavy tailed sample distributions. It covers the estimation of the mean of a vector or a matrix, with applications to least squares linear regression. Special efforts are devoted to the estimation of Gram matrices, due to their prominent role in high-dimension data analysis.