How Convolutional Neural Networks Deal with Aliasing

TL;DR

This paper investigates how convolutional neural networks handle aliasing caused by downsampling, revealing that while they have some capacity to distinguish high-frequency signals, they do not explicitly prevent aliasing in their layers.

Contribution

The study provides empirical analysis of aliasing in CNNs, showing that intermediate redundancies aid in distinguishing oscillations but CNNs do not explicitly implement anti-aliasing mechanisms.

Findings

Redundancies in intermediate channels help distinguish oscillations.

CNNs do not explicitly prevent aliasing in intermediate layers.

CNNs have some capacity to handle high-frequency components.

Abstract

The convolutional neural network (CNN) remains an essential tool in solving computer vision problems. Standard convolutional architectures consist of stacked layers of operations that progressively downscale the image. Aliasing is a well-known side-effect of downsampling that may take place: it causes high-frequency components of the original signal to become indistinguishable from its low-frequency components. While downsampling takes place in the max-pooling layers or in the strided-convolutions in these models, there is no explicit mechanism that prevents aliasing from taking place in these layers. Due to the impressive performance of these models, it is natural to suspect that they, somehow, implicitly deal with this distortion. The question we aim to answer in this paper is simply: "how and to what extent do CNNs counteract aliasing?" We explore the question by means of two examples: In the first, we assess the CNNs capability of distinguishing oscillations at the input, showing that the redundancies in the intermediate channels play an important role in succeeding at the task; In the second, we show that an image classifier CNN while, in principle, capable of implementing anti-aliasing filters, does not prevent aliasing from taking place in the intermediate layers.