The Overfocusing Bias of Convolutional Neural Networks: A Saliency-Guided Regularization Approach

TL;DR

This paper identifies the overfocusing bias in CNNs, especially in low-data scenarios, and proposes a saliency-guided regularization method called SGDrop to promote broader feature attention and improve generalization.

Contribution

Introduces SGDrop, a novel regularization technique that uses attribution methods to diversify CNN attention and enhance performance in limited data settings.

Findings

SGDrop improves CNN generalization across benchmarks.

Models with SGDrop show more expansive attributions.

SGDrop reduces overfocusing on specific image regions.

Abstract

Despite transformers being considered as the new standard in computer vision, convolutional neural networks (CNNs) still outperform them in low-data regimes. Nonetheless, CNNs often make decisions based on narrow, specific regions of input images, especially when training data is limited. This behavior can severely compromise the model's generalization capabilities, making it disproportionately dependent on certain features that might not represent the broader context of images. While the conditions leading to this phenomenon remain elusive, the primary intent of this article is to shed light on this observed behavior of neural networks. Our research endeavors to prioritize comprehensive insight and to outline an initial response to this phenomenon. In line with this, we introduce Saliency Guided Dropout (SGDrop), a pioneering regularization approach tailored to address this specific issue. SGDrop utilizes attribution methods on the feature map to identify and then reduce the influence of the most salient features during training. This process encourages the network to diversify its attention and not focus solely on specific standout areas. Our experiments across several visual classification benchmarks validate SGDrop's role in enhancing generalization. Significantly, models incorporating SGDrop display more expansive attributions and neural activity, offering a more comprehensive view of input images in contrast to their traditionally trained counterparts.