PAC-Bayesian bounds for the Gram matrix and least squares regression with a random design

TL;DR

This paper develops PAC-Bayesian bounds for estimating the Gram matrix and covariance matrix, introduces robust estimators for these matrices and least squares regression, and analyzes their convergence rates under moment assumptions.

Contribution

It presents novel PAC-Bayesian bounds and robust estimators for the Gram matrix, covariance matrix, and least squares regression with a random design, including convergence rates.

Findings

A new robust estimator of the Gram matrix and covariance matrix.

Results on the empirical Gram matrix and least squares estimators.

Exact convergence rates for the ordinary least squares estimator under polynomial moments.

Abstract

The topics dicussed in this paper take their origin inthe estimation of the Gram matrix of a random vector from a sample made of n independent copies. They comprise the estimation of the covariance matrix and the study of least squares regression with a random design. We propose four types of results, based on non-asymptotic PAC-Bayesian generalization bounds: a new robust estimator of the Gram matrix and of the covariance matrix, new results on the empirical Gram matrix, new robust least squares estimators and new results on the ordinary least squares estimator, including its exact rate of convergence under polynomial moment assumptions.