Scaling Out-of-Distribution Detection for Real-World Settings

TL;DR

This paper advances out-of-distribution detection by creating large-scale benchmarks and demonstrating that a simple maximum logit detector outperforms complex prior methods across various realistic, high-resolution, multi-class, multi-label, and segmentation tasks.

Contribution

It introduces new large-scale benchmarks for out-of-distribution detection in realistic settings and shows that a simple maximum logit method outperforms existing complex approaches.

Findings

Simple maximum logit detector outperforms prior methods

New benchmarks for large-scale OOD detection are established

Simple baseline sets a new standard for future research

Abstract

Detecting out-of-distribution examples is important for safety-critical machine learning applications such as detecting novel biological phenomena and self-driving cars. However, existing research mainly focuses on simple small-scale settings. To set the stage for more realistic out-of-distribution detection, we depart from small-scale settings and explore large-scale multiclass and multi-label settings with high-resolution images and thousands of classes. To make future work in real-world settings possible, we create new benchmarks for three large-scale settings. To test ImageNet multiclass anomaly detectors, we introduce the Species dataset containing over 700,000 images and over a thousand anomalous species. We leverage ImageNet-21K to evaluate PASCAL VOC and COCO multilabel anomaly detectors. Third, we introduce a new benchmark for anomaly segmentation by introducing a segmentation benchmark with road anomalies. We conduct extensive experiments in these more realistic settings for out-of-distribution detection and find that a surprisingly simple detector based on the maximum logit outperforms prior methods in all the large-scale multi-class, multi-label, and segmentation tasks, establishing a simple new baseline for future work.