Why do CNNs excel at feature extraction? A mathematical explanation

TL;DR

This paper provides a mathematical explanation for why CNNs excel at feature extraction, demonstrating that they can solve image classification tasks with zero error by modeling feature detection with piecewise linear functions.

Contribution

It introduces a novel mathematical model for image classification based on feature extraction, showing CNNs can realize these models with zero error.

Findings

CNNs can solve feature-based classification tasks with zero error.

Piecewise linear functions can be implemented by CNNs for feature detection.

Theoretical proof connects CNNs' structure to their effectiveness in feature extraction.

Abstract

Over the past decade deep learning has revolutionized the field of computer vision, with convolutional neural network models proving to be very effective for image classification benchmarks. However, a fundamental theoretical questions remain answered: why can they solve discrete image classification tasks that involve feature extraction? We address this question in this paper by introducing a novel mathematical model for image classification, based on feature extraction, that can be used to generate images resembling real-world datasets. We show that convolutional neural network classifiers can solve these image classification tasks with zero error. In our proof, we construct piecewise linear functions that detect the presence of features, and show that they can be realized by a convolutional network.