Pixel-wise Anomaly Detection in Complex Driving Scenes

TL;DR

This paper introduces a pixel-wise anomaly detection framework for complex driving scenes that combines segmentation uncertainty and image re-synthesis to improve anomaly detection without affecting segmentation accuracy.

Contribution

The authors propose a novel framework that integrates uncertainty maps with re-synthesis methods, enhancing anomaly detection in autonomous driving scenarios.

Findings

Outperforms existing methods in anomaly detection accuracy

Works with pre-trained segmentation networks without loss of segmentation performance

Achieves top-2 results across multiple anomaly datasets

Abstract

The inability of state-of-the-art semantic segmentation methods to detect anomaly instances hinders them from being deployed in safety-critical and complex applications, such as autonomous driving. Recent approaches have focused on either leveraging segmentation uncertainty to identify anomalous areas or re-synthesizing the image from the semantic label map to find dissimilarities with the input image. In this work, we demonstrate that these two methodologies contain complementary information and can be combined to produce robust predictions for anomaly segmentation. We present a pixel-wise anomaly detection framework that uses uncertainty maps to improve over existing re-synthesis methods in finding dissimilarities between the input and generated images. Our approach works as a general framework around already trained segmentation networks, which ensures anomaly detection without compromising segmentation accuracy, while significantly outperforming all similar methods. Top-2 performance across a range of different anomaly datasets shows the robustness of our approach to handling different anomaly instances.