On the Blindspots of Convolutional Networks

TL;DR

This paper reveals limitations of convolutional networks by introducing signals they cannot detect, demonstrating that traditional models can outperform them when these signals are present, thus highlighting their blind spots.

Contribution

The study systematically analyzes convolutional networks' blind spots using designed signals, providing insights into their limitations and potential improvements.

Findings

Convolutional networks miss certain signals detectable by simpler models.

Injecting signals reveals the weaknesses of convolutional architectures.

Traditional models can outperform CNNs when signals are designed to exploit their blind spots.

Abstract

Deep convolutional network has been the state-of-the-art approach for a wide variety of tasks over the last few years. Its successes have, in many cases, turned it into the default model in quite a few domains. In this work, we will demonstrate that convolutional networks have limitations that may, in some cases, hinder it from learning properties of the data, which are easily recognizable by traditional, less demanding, models. To this end, we present a series of competitive analysis studies on image recognition and text analysis tasks, for which convolutional networks are known to provide state-of-the-art results. In our studies, we inject a truth-revealing signal, indiscernible for the network, thus hitting time and again the network's blind spots. The signal does not impair the network's existing performances, but it does provide an opportunity for a significant performance boost by models that can capture it. The various forms of the carefully designed signals shed a light on the strengths and weaknesses of convolutional network, which may provide insights for both theoreticians that study the power of deep architectures, and for practitioners that consider applying convolutional networks to the task at hand.