Anomaly-Aware Semantic Segmentation by Leveraging Synthetic-Unknown Data

TL;DR

This paper introduces a novel approach for anomaly-aware semantic segmentation using synthetic-unknown data generation and a Masked Gradient Update module, achieving state-of-the-art results in anomaly segmentation tasks.

Contribution

It proposes a new synthetic-unknown data generation method and a Masked Gradient Update module to improve anomaly-aware semantic segmentation.

Findings

Achieved state-of-the-art performance on two anomaly segmentation datasets.

Demonstrated the effectiveness of the proposed modules through ablation studies.

Enhanced boundary data emphasis improves segmentation accuracy.

Abstract

Anomaly awareness is an essential capability for safety-critical applications such as autonomous driving. While recent progress of robotics and computer vision has enabled anomaly detection for image classification, anomaly detection on semantic segmentation is less explored. Conventional anomaly-aware systems assuming other existing classes as out-of-distribution (pseudo-unknown) classes for training a model will result in two drawbacks. (1) Unknown classes, which applications need to cope with, might not actually exist during training time. (2) Model performance would strongly rely on the class selection. Observing this, we propose a novel Synthetic-Unknown Data Generation, intending to tackle the anomaly-aware semantic segmentation task. We design a new Masked Gradient Update (MGU) module to generate auxiliary data along the boundary of in-distribution data points. In addition, we modify the traditional cross-entropy loss to emphasize the border data points. We reach the state-of-the-art performance on two anomaly segmentation datasets. Ablation studies also demonstrate the effectiveness of proposed modules.