Learning Robust Global Representations by Penalizing Local Predictive Power

TL;DR

This paper introduces a method to improve the robustness of convolutional neural networks by penalizing local representations, encouraging reliance on global image structures, which enhances cross-domain generalization.

Contribution

The paper proposes a novel training approach that penalizes local predictive power in CNNs to promote global feature reliance, improving domain adaptation performance.

Findings

Enhanced generalization on synthetic and benchmark domain adaptation tasks

Improved cross-domain transferability of CNNs

Introduction of ImageNet-Sketch dataset for evaluation

Abstract

Despite their renowned predictive power on i.i.d. data, convolutional neural networks are known to rely more on high-frequency patterns that humans deem superficial than on low-frequency patterns that agree better with intuitions about what constitutes category membership. This paper proposes a method for training robust convolutional networks by penalizing the predictive power of the local representations learned by earlier layers. Intuitively, our networks are forced to discard predictive signals such as color and texture that can be gleaned from local receptive fields and to rely instead on the global structures of the image. Across a battery of synthetic and benchmark domain adaptation tasks, our method confers improved generalization out of the domain. Also, to evaluate cross-domain transfer, we introduce ImageNet-Sketch, a new dataset consisting of sketch-like images, that matches the ImageNet classification validation set in categories and scale.