Deep Convolutional Neural Networks with Zero-Padding: Feature Extraction and Learning

TL;DR

This paper demonstrates that deep convolutional neural networks with zero-padding excel in feature extraction and learning, offering translation-invariance and universal consistency, supported by theoretical analysis and numerical experiments.

Contribution

It shows that DCNNs with zero-padding can represent fully connected networks with similar parameters and are inherently better for feature extraction and translation-invariant learning.

Findings

DCNNs with zero-padding enable translation-equivalence.

DCNNs with zero-padding can represent any DFCN with similar parameters.

Theoretical results are validated by numerical experiments.

Abstract

This paper studies the performance of deep convolutional neural networks (DCNNs) with zero-padding in feature extraction and learning. After verifying the roles of zero-padding in enabling translation-equivalence, and pooling in its translation-invariance driven nature, we show that with similar number of free parameters, any deep fully connected networks (DFCNs) can be represented by DCNNs with zero-padding. This demonstrates that DCNNs with zero-padding is essentially better than DFCNs in feature extraction. Consequently, we derive universal consistency of DCNNs with zero-padding and show its translation-invariance in the learning process. All our theoretical results are verified by numerical experiments including both toy simulations and real-data running.