Towards Robust Semantic Segmentation of Accident Scenes via Multi-Source Mixed Sampling and Meta-Learning

TL;DR

This paper introduces a novel multi-source meta-learning framework with mixed sampling and advanced decoding techniques to improve the robustness of semantic segmentation models in accident scenarios, significantly outperforming previous methods.

Contribution

The paper proposes MMUDA, a multi-source meta-learning unsupervised domain adaptation framework with hybrid decoding, enhancing segmentation robustness in abnormal accident scenes.

Findings

Achieves 46.97% mIoU on DADA-seg benchmark, surpassing previous state-of-the-art by 7.50%.

Utilizes multi-source mixed sampling to augment training data with abnormal scene appearances.

Incorporates a hybrid decoder with large window attention and strip pooling for better contextual understanding.

Abstract

Autonomous vehicles utilize urban scene segmentation to understand the real world like a human and react accordingly. Semantic segmentation of normal scenes has experienced a remarkable rise in accuracy on conventional benchmarks. However, a significant portion of real-life accidents features abnormal scenes, such as those with object deformations, overturns, and unexpected traffic behaviors. Since even small mis-segmentation of driving scenes can lead to serious threats to human lives, the robustness of such models in accident scenarios is an extremely important factor in ensuring safety of intelligent transportation systems. In this paper, we propose a Multi-source Meta-learning Unsupervised Domain Adaptation (MMUDA) framework, to improve the generalization of segmentation transformers to extreme accident scenes. In MMUDA, we make use of Multi-Domain Mixed Sampling to augment the images of multiple-source domains (normal scenes) with the target data appearances (abnormal scenes). To train our model, we intertwine and study a meta-learning strategy in the multi-source setting for robustifying the segmentation results. We further enhance the segmentation backbone (SegFormer) with a HybridASPP decoder design, featuring large window attention spatial pyramid pooling and strip pooling, to efficiently aggregate long-range contextual dependencies. Our approach achieves a mIoU score of 46.97% on the DADA-seg benchmark, surpassing the previous state-of-the-art model by more than 7.50%. Code will be made publicly available at https://github.com/xinyu-laura/MMUDA.