Robustness of Humans and Machines on Object Recognition with Extreme Image Transformations

TL;DR

This paper compares human and neural network robustness in object recognition under extreme image transformations, revealing humans outperform networks significantly in challenging conditions.

Contribution

It introduces new image transforms to evaluate and compare human and neural network object recognition robustness under extreme distortions.

Findings

Humans maintain high recognition accuracy despite extreme image distortions.

Neural networks' performance drops rapidly with extreme transformations.

Humans and networks use fundamentally different strategies for visual recognition.

Abstract

Recent neural network architectures have claimed to explain data from the human visual cortex. Their demonstrated performance is however still limited by the dependence on exploiting low-level features for solving visual tasks. This strategy limits their performance in case of out-of-distribution/adversarial data. Humans, meanwhile learn abstract concepts and are mostly unaffected by even extreme image distortions. Humans and networks employ strikingly different strategies to solve visual tasks. To probe this, we introduce a novel set of image transforms and evaluate humans and networks on an object recognition task. We found performance for a few common networks quickly decreases while humans are able to recognize objects with a high accuracy.