Optimal Approximation and Learning Rates for Deep Convolutional Neural Networks

TL;DR

This paper analyzes the approximation and learning capabilities of deep convolutional neural networks, establishing near-optimal rates for approximating smooth functions and for empirical risk minimization.

Contribution

It provides theoretical proofs of approximation and learning rates for deep CNNs, showing they are nearly optimal up to a logarithmic factor.

Findings

Approximation rates for smooth functions are of order (L^2 / log L)^{-2r/d}.

Deep CNNs achieve almost optimal learning rates for empirical risk minimization.

The results are applicable to CNNs with zero-padding and max-pooling.

Abstract

This paper focuses on approximation and learning performance analysis for deep convolutional neural networks with zero-padding and max-pooling. We prove that, to approximate $r$-smooth function, the approximation rates of deep convolutional neural networks with depth $L$ are of order $ (L^2/\log L)^{-2r/d} $, which is optimal up to a logarithmic factor. Furthermore, we deduce almost optimal learning rates for implementing empirical risk minimization over deep convolutional neural networks.