The Invisible Gorilla Effect in Out-of-distribution Detection

TL;DR

This paper uncovers a bias in out-of-distribution detection where detection performance improves when artefacts resemble the model's ROI, revealing a failure mode called the Invisible Gorilla Effect that impacts robustness.

Contribution

It identifies the Invisible Gorilla Effect in OOD detection, demonstrating how artefact similarity influences detection performance and providing a comprehensive evaluation across multiple methods and datasets.

Findings

Detection improves with artefact similarity to ROI

Most OOD methods show performance drops on dissimilar artefacts

Annotated artefacts and generated counterfactuals reveal bias in detection methods

Abstract

Deep Neural Networks achieve high performance in vision tasks by learning features from regions of interest (ROI) within images, but their performance degrades when deployed on out-of-distribution (OOD) data that differs from training data. This challenge has led to OOD detection methods that aim to identify and reject unreliable predictions. Although prior work shows that OOD detection performance varies by artefact type, the underlying causes remain underexplored. To this end, we identify a previously unreported bias in OOD detection: for hard-to-detect artefacts (near-OOD), detection performance typically improves when the artefact shares visual similarity (e.g. colour) with the model's ROI and drops when it does not - a phenomenon we term the Invisible Gorilla Effect. For example, in a skin lesion classifier with red lesion ROI, we show the method Mahalanobis Score achieves a 31.5% higher AUROC when detecting OOD red ink (similar to ROI) compared to black ink (dissimilar) annotations. We annotated artefacts by colour in 11,355 images from three public datasets (e.g. ISIC) and generated colour-swapped counterfactuals to rule out dataset bias. We then evaluated 40 OOD methods across 7 benchmarks and found significant performance drops for most methods when artefacts differed from the ROI. Our findings highlight an overlooked failure mode in OOD detection and provide guidance for more robust detectors. Code and annotations are available at: https://github.com/HarryAnthony/Invisible_Gorilla_Effect.