Assessing The Importance Of Colours For CNNs In Object Recognition

TL;DR

This study empirically explores the role of colours in CNN-based object recognition, revealing that CNNs often depend heavily on colour cues, with dependency varying across datasets and training methods.

Contribution

It provides the first comprehensive analysis of colour importance in CNNs for object recognition, highlighting how training from scratch increases colour reliance and pre-training reduces it.

Findings

CNNs rely heavily on colour information for predictions.

Dependency on colour varies across datasets.

Pre-training reduces colour dependence.

Abstract

Humans rely heavily on shapes as a primary cue for object recognition. As secondary cues, colours and textures are also beneficial in this regard. Convolutional neural networks (CNNs), an imitation of biological neural networks, have been shown to exhibit conflicting properties. Some studies indicate that CNNs are biased towards textures whereas, another set of studies suggests shape bias for a classification task. However, they do not discuss the role of colours, implying its possible humble role in the task of object recognition. In this paper, we empirically investigate the importance of colours in object recognition for CNNs. We are able to demonstrate that CNNs often rely heavily on colour information while making a prediction. Our results show that the degree of dependency on colours tend to vary from one dataset to another. Moreover, networks tend to rely more on colours if trained from scratch. Pre-training can allow the model to be less colour dependent. To facilitate these findings, we follow the framework often deployed in understanding role of colours in object recognition for humans. We evaluate a model trained with congruent images (images in original colours eg. red strawberries) on congruent, greyscale, and incongruent images (images in unnatural colours eg. blue strawberries). We measure and analyse network's predictive performance (top-1 accuracy) under these different stylisations. We utilise standard datasets of supervised image classification and fine-grained image classification in our experiments.