Abutting Grating Illusion: Cognitive Challenge to Neural Network Models

TL;DR

This study introduces a novel visual corruption based on the abutting grating illusion to evaluate deep learning models' robustness, revealing significant challenges for models and insights into neural network properties compared to human perception.

Contribution

The paper proposes a new corruption method inspired by human visual illusions and evaluates its impact on various deep learning models, highlighting their limitations and potential improvements.

Findings

Most models perform poorly on abutting grating corruption

DeepAugment improves robustness against the illusion

Better models show stronger end-stopping properties

Abstract

Even the state-of-the-art deep learning models lack fundamental abilities compared to humans. Multiple comparison paradigms have been proposed to explore the distinctions between humans and deep learning. While most comparisons rely on corruptions inspired by mathematical transformations, very few have bases on human cognitive phenomena. In this study, we propose a novel corruption method based on the abutting grating illusion, which is a visual phenomenon widely discovered in both human and a wide range of animal species. The corruption method destroys the gradient-defined boundaries and generates the perception of illusory contours using line gratings abutting each other. We applied the method on MNIST, high resolution MNIST, and silhouette object images. Various deep learning models are tested on the corruption, including models trained from scratch and 109 models pretrained with ImageNet or various data augmentation techniques. Our results show that abutting grating corruption is challenging even for state-of-the-art deep learning models because most models are randomly guessing. We also discovered that the DeepAugment technique can greatly improve robustness against abutting grating illusion. Visualisation of early layers indicates that better performing models exhibit stronger end-stopping property, which is consistent with neuroscience discoveries. To validate the corruption method, 24 human subjects are involved to classify samples of corrupted datasets.