Self-Supervised Likelihood Estimation with Energy Guidance for Anomaly Segmentation in Urban Scenes

TL;DR

This paper introduces a self-supervised energy-guided framework for anomaly segmentation in urban scenes, leveraging contextual information and likelihood estimation to improve detection without auxiliary data.

Contribution

It proposes a novel energy-guided self-supervised approach with two anomaly likelihood estimators, enhancing anomaly segmentation accuracy without requiring auxiliary data.

Findings

Achieves competitive results on Fishyscapes and Road Anomaly benchmarks.

Does not rely on auxiliary data or synthetic models.

Refines anomaly masks using context-dependent likelihood estimation.

Abstract

Robust autonomous driving requires agents to accurately identify unexpected areas (anomalies) in urban scenes. To this end, some critical issues remain open: how to design advisable metric to measure anomalies, and how to properly generate training samples of anomaly data? Classical effort in anomaly detection usually resorts to pixel-wise uncertainty or sample synthesis, which ignores the contextual information and sometimes requires auxiliary data with fine-grained annotations. On the contrary, in this paper, we exploit the strong context-dependent nature of the segmentation task and design an energy-guided self-supervised framework for anomaly segmentation, which optimizes an anomaly head by maximizing the likelihood of self-generated anomaly pixels. For this purpose, we design two estimators to model anomaly likelihood, one is a task-agnostic binary estimator and the other depicts the likelihood as residual of task-oriented joint energy. Based on the proposed estimators, we devise an adaptive self-supervised training framework, which exploits the contextual reliance and estimated likelihood to refine mask annotations in anomaly areas. We conduct extensive experiments on challenging Fishyscapes and Road Anomaly benchmarks, demonstrating that without any auxiliary data or synthetic models, our method can still achieve comparable performance to supervised competitors. Code is available at https://github.com/yuanpengtu/SLEEG..