On the Limitation of Convolutional Neural Networks in Recognizing Negative Images

TL;DR

This paper investigates the limitations of CNNs in understanding the semantics of images by testing their performance on negative images, revealing significant generalization issues and introducing the concept of semantic adversarial examples.

Contribution

The study demonstrates CNNs' inability to generalize semantics to negative images and introduces semantic adversarial examples as a new challenge in model robustness.

Findings

CNN accuracy drops significantly on negative images compared to regular images.

Current training methods do not effectively teach models to understand semantic content.

Negative images are identified as a class of semantic adversarial examples.

Abstract

Convolutional Neural Networks (CNNs) have achieved state-of-the-art performance on a variety of computer vision tasks, particularly visual classification problems, where new algorithms reported to achieve or even surpass the human performance. In this paper, we examine whether CNNs are capable of learning the semantics of training data. To this end, we evaluate CNNs on negative images, since they share the same structure and semantics as regular images and humans can classify them correctly. Our experimental results indicate that when training on regular images and testing on negative images, the model accuracy is significantly lower than when it is tested on regular images. This leads us to the conjecture that current training methods do not effectively train models to generalize the concepts. We then introduce the notion of semantic adversarial examples - transformed inputs that semantically represent the same objects, but the model does not classify them correctly - and present negative images as one class of such inputs.

Full text

See pages 1-last of Main.pdf