Toward a Taxonomy and Computational Models of Abnormalities in Images

TL;DR

This paper introduces a new dataset and taxonomy for abnormalities in images, along with a computational model that effectively recognizes various types of atypicalities, advancing the understanding of visual anomalies.

Contribution

It presents a comprehensive taxonomy of image abnormalities and a novel computational model that outperforms previous methods in abnormality detection.

Findings

Three major categories of abnormalities: object-centric, scene-centric, and contextual.

A new dataset of diverse abnormal images.

The proposed model outperforms prior approaches in abnormality recognition.

Abstract

The human visual system can spot an abnormal image, and reason about what makes it strange. This task has not received enough attention in computer vision. In this paper we study various types of atypicalities in images in a more comprehensive way than has been done before. We propose a new dataset of abnormal images showing a wide range of atypicalities. We design human subject experiments to discover a coarse taxonomy of the reasons for abnormality. Our experiments reveal three major categories of abnormality: object-centric, scene-centric, and contextual. Based on this taxonomy, we propose a comprehensive computational model that can predict all different types of abnormality in images and outperform prior arts in abnormality recognition.