Random Walk on Pixel Manifolds for Anomaly Segmentation of Complex Driving Scenes

TL;DR

This paper introduces RWPM, a method that uses random walks on pixel manifolds to refine embeddings, significantly improving anomaly segmentation accuracy in complex driving scenes.

Contribution

The paper proposes a novel random walk approach on pixel manifolds to address embedding distortion, enhancing anomaly score prediction in driving scene segmentation.

Findings

RWPM improves anomaly segmentation performance across multiple benchmarks.

Refined pixel embeddings lead to more accurate inlier logit predictions.

The method achieves state-of-the-art results in complex driving scene analysis.

Abstract

In anomaly segmentation for complex driving scenes, state-of-the-art approaches utilize anomaly scoring functions to calculate anomaly scores. For these functions, accurately predicting the logits of inlier classes for each pixel is crucial for precisely inferring the anomaly score. However, in real-world driving scenarios, the diversity of scenes often results in distorted manifolds of pixel embeddings in the space. This effect is not conducive to directly using the pixel embeddings for the logit prediction during inference, a concern overlooked by existing methods. To address this problem, we propose a novel method called Random Walk on Pixel Manifolds (RWPM). RWPM utilizes random walks to reveal the intrinsic relationships among pixels to refine the pixel embeddings. The refined pixel embeddings alleviate the distortion of manifolds, improving the accuracy of anomaly scores. Our extensive experiments show that RWPM consistently improve the performance of the existing anomaly segmentation methods and achieve the best results. Code is available at: \url{https://github.com/ZelongZeng/RWPM}.