Do you see what I see? An Ambiguous Optical Illusion Dataset exposing limitations of Explainable AI

TL;DR

This paper introduces a novel optical illusion dataset with ambiguous images to evaluate and improve the robustness of machine learning models, highlighting perceptual biases and limitations in visual understanding.

Contribution

The work presents a new optical illusion dataset with systematically generated illusions to study model perception and bias, addressing a gap in existing datasets.

Findings

Models are significantly affected by perceptual ambiguity.

Visual concepts like gaze direction influence model accuracy.

The dataset reveals limitations of current AI in understanding ambiguous visuals.

Abstract

From uncertainty quantification to real-world object detection, we recognize the importance of machine learning algorithms, particularly in safety-critical domains such as autonomous driving or medical diagnostics. In machine learning, ambiguous data plays an important role in various machine learning domains. Optical illusions present a compelling area of study in this context, as they offer insight into the limitations of both human and machine perception. Despite this relevance, optical illusion datasets remain scarce. In this work, we introduce a novel dataset of optical illusions featuring intermingled animal pairs designed to evoke perceptual ambiguity. We identify generalizable visual concepts, particularly gaze direction and eye cues, as subtle yet impactful features that significantly influence model accuracy. By confronting models with perceptual ambiguity, our findings underscore the importance of concepts in visual learning and provide a foundation for studying bias and alignment between human and machine vision. To make this dataset useful for general purposes, we generate optical illusions systematically with different concepts discussed in our bias mitigation section. The dataset is accessible in Kaggle via https://kaggle.com/datasets/693bf7c6dd2cb45c8a863f9177350c8f9849a9508e9d50526e2ffcc5559a8333. Our source code can be found at https://github.com/KDD-OpenSource/Ambivision.git.