Quadratic Suffices for Over-parametrization via Matrix Chernoff Bound

TL;DR

This paper advances the theoretical understanding of over-parametrization in deep learning by providing improved bounds using matrix Chernoff bounds, enhancing previous results in the field.

Contribution

It introduces tighter over-parametrization bounds in deep learning theory through the application of matrix Chernoff bounds, improving upon prior results.

Findings

Enhanced over-parametrization bounds in deep learning

Application of matrix Chernoff bounds to neural network theory

Improved theoretical guarantees for model capacity

Abstract

We improve the over-parametrization size over two beautiful results [Li and Liang' 2018] and [Du, Zhai, Poczos and Singh' 2019] in deep learning theory.