Anomaly Discovery in Semantic Segmentation via Distillation Comparison Networks

TL;DR

This paper introduces DiCNet, a novel approach for anomaly discovery in semantic segmentation that leverages distillation-based feature discrepancy to improve detection accuracy and reduce false positives.

Contribution

The paper proposes a distillation comparison network that enhances anomaly detection by comparing semantic features from a teacher and student branch, avoiding reliance on semantic classification.

Findings

6.3% improvement in AUPR on StreetHazards

4.2% improvement in AUPR on BDD-Anomaly

Significant reduction in false positive rate

Abstract

This paper aims to address the problem of anomaly discovery in semantic segmentation. Our key observation is that semantic classification plays a critical role in existing approaches, while the incorrectly classified pixels are easily regarded as anomalies. Such a phenomenon frequently appears and is rarely discussed, which significantly reduces the performance of anomaly discovery. To this end, we propose a novel Distillation Comparison Network (DiCNet). It comprises of a teacher branch which is a semantic segmentation network that removed the semantic classification head, and a student branch that is distilled from the teacher branch through a distribution distillation. We show that the distillation guarantees the semantic features of the two branches hold consistency in the known classes, while reflect inconsistency in the unknown class. Therefore, we leverage the semantic feature discrepancy between the two branches to discover the anomalies. DiCNet abandons the semantic classification head in the inference process, and hence significantly alleviates the issue caused by incorrect semantic classification. Extensive experimental results on StreetHazards dataset and BDD-Anomaly dataset are conducted to verify the superior performance of DiCNet. In particular, DiCNet obtains a 6.3% improvement in AUPR and a 5.2% improvement in FPR95 on StreetHazards dataset, achieves a 4.2% improvement in AUPR and a 6.8% improvement in FPR95 on BDD-Anomaly dataset. Codes are available at https://github.com/zhouhuan-hust/DiCNet.