A Note on the Regularity of Images Generated by Convolutional Neural Networks

TL;DR

This paper investigates the smoothness properties of images generated by convolutional neural networks, revealing they are inherently continuous and sometimes differentiable, which challenges the common assumption of sharp edges in such images.

Contribution

It provides a theoretical analysis showing that in an infinite-dimensional setting, CNN-generated images are always continuous, offering insights into regularization effects and image regularity.

Findings

Generated images are always continuous in the infinite-dimensional limit.

Under certain conditions, images are also continuously differentiable.

L2 regularization may lead to overly smooth, over-smoothed images.

Abstract

The regularity of images generated by convolutional neural networks, such as the U-net, generative networks, or the deep image prior, is analyzed. In a resolution-independent, infinite dimensional setting, it is shown that such images, represented as functions, are always continuous and, in some circumstances, even continuously differentiable, contradicting the widely accepted modeling of sharp edges in images via jump discontinuities. While such statements require an infinite dimensional setting, the connection to (discretized) neural networks used in practice is made by considering the limit as the resolution approaches infinity. As practical consequence, the results of this paper in particular provide analytical evidence that basic L2 regularization of network weights might lead to over-smoothed outputs.