Towards Robust Unsupervised Attention Prediction in Autonomous Driving

TL;DR

This paper introduces a robust unsupervised attention prediction method for autonomous driving that enhances safety by effectively handling domain gaps, weather adversities, and traffic complexities, outperforming supervised methods on multiple benchmarks.

Contribution

The paper proposes a novel unsupervised attention prediction framework with an Uncertainty Mining Branch, Knowledge Embedding Block, and RoboMixup data augmentation, significantly improving robustness and domain adaptation in autonomous driving scenarios.

Findings

Achieves state-of-the-art performance on public datasets.

Reduces corruption degradation by over 50%.

Enhances central bias robustness by around 12%.

Abstract

Robustly predicting attention regions of interest for self-driving systems is crucial for driving safety but presents significant challenges due to the labor-intensive nature of obtaining large-scale attention labels and the domain gap between self-driving scenarios and natural scenes. These challenges are further exacerbated by complex traffic environments, including camera corruption under adverse weather, noise interferences, and central bias from long-tail distributions. To address these issues, we propose a robust unsupervised attention prediction method. An Uncertainty Mining Branch refines predictions by analyzing commonalities and differences across multiple pre-trained models on natural scenes, while a Knowledge Embedding Block bridges the domain gap by incorporating driving knowledge to adaptively enhance pseudo-labels. Additionally, we introduce RoboMixup, a novel data augmentation method that improves robustness against corruption through soft attention and dynamic augmentation, and mitigates central bias by integrating random cropping into Mixup as a regularizer. To systematically evaluate robustness in self-driving attention prediction, we introduce the DriverAttention-C benchmark, comprising over 100k frames across three subsets: BDD-A-C, DR(eye)VE-C, and DADA-2000-C. Our method achieves performance equivalent to or surpassing fully supervised state-of-the-art approaches on three public datasets and the proposed robustness benchmark, reducing relative corruption degradation by 58.8% and 52.8%, and improving central bias robustness by 12.4% and 11.4% in KLD and CC metrics, respectively. Code and data are available at https://github.com/zaplm/DriverAttention.