Camera Bias in a Fine Grained Classification Task

TL;DR

This paper demonstrates that CNNs can exploit camera-specific features to cheat in fine-grained classification tasks, leading to poor generalization when camera correlations are absent or unencountered.

Contribution

The study reveals the extent of camera bias in CNNs for fine-grained classification and investigates the visual features exploited, highlighting issues with dataset biases.

Findings

Models rely on high frequency features rather than global color statistics.

Camera/label correlations cause poor generalization to new cameras.

Image processing algorithms may introduce exploitable high frequency features.

Abstract

We show that correlations between the camera used to acquire an image and the class label of that image can be exploited by convolutional neural networks (CNN), resulting in a model that "cheats" at an image classification task by recognizing which camera took the image and inferring the class label from the camera. We show that models trained on a dataset with camera / label correlations do not generalize well to images in which those correlations are absent, nor to images from unencountered cameras. Furthermore, we investigate which visual features they are exploiting for camera recognition. Our experiments present evidence against the importance of global color statistics, lens deformation and chromatic aberration, and in favor of high frequency features, which may be introduced by image processing algorithms built into the cameras.