Signal Strength and Noise Drive Feature Preference in CNN Image Classifiers

TL;DR

This study investigates how CNN image classifiers prioritize features like shape, texture, and color based on signal strength and noise, revealing preferences that can inform bias mitigation and comparison with human vision.

Contribution

It demonstrates that CNNs prefer features with higher signal and lower noise, regardless of feature type, providing a predictive model for feature preference and highlighting bias pathways.

Findings

CNNs prefer features with stronger signal and lower noise

Preferences are consistent across shape, texture, and color

Experimental setup influences feature bias

Abstract

Feature preference in Convolutional Neural Network (CNN) image classifiers is integral to their decision making process, and while the topic has been well studied, it is still not understood at a fundamental level. We test a range of task relevant feature attributes (including shape, texture, and color) with varying degrees of signal and noise in highly controlled CNN image classification experiments using synthetic datasets to determine feature preferences. We find that CNNs will prefer features with stronger signal strength and lower noise irrespective of whether the feature is texture, shape, or color. This provides guidance for a predictive model for task relevant feature preferences, demonstrates pathways for bias in machine models that can be avoided with careful controls on experimental setup, and suggests that comparisons between how humans and machines prefer task relevant features in vision classification tasks should be revisited. Code to reproduce experiments in this paper can be found at \url{https://github.com/mwolff31/signal_preference}.