Predicting Driver Attention in Critical Situations

TL;DR

This paper introduces a new in-lab driver attention dataset and a human-weighted sampling method, leading to a model that accurately predicts driver attention in critical situations, outperforming existing methods.

Contribution

The paper presents a novel in-lab data collection protocol, a new attention dataset, and a human-weighted sampling technique that improves driver attention prediction models.

Findings

The model outperforms state-of-the-art in driver attention prediction.

It accurately attends to crossing pedestrians without false alarms.

It predicts in-car driver attention with high accuracy using only in-lab data.

Abstract

Robust driver attention prediction for critical situations is a challenging computer vision problem, yet essential for autonomous driving. Because critical driving moments are so rare, collecting enough data for these situations is difficult with the conventional in-car data collection protocol---tracking eye movements during driving. Here, we first propose a new in-lab driver attention collection protocol and introduce a new driver attention dataset, Berkeley DeepDrive Attention (BDD-A) dataset, which is built upon braking event videos selected from a large-scale, crowd-sourced driving video dataset. We further propose Human Weighted Sampling (HWS) method, which uses human gaze behavior to identify crucial frames of a driving dataset and weights them heavily during model training. With our dataset and HWS, we built a driver attention prediction model that outperforms the state-of-the-art and demonstrates sophisticated behaviors, like attending to crossing pedestrians but not giving false alarms to pedestrians safely walking on the sidewalk. Its prediction results are nearly indistinguishable from ground-truth to humans. Although only being trained with our in-lab attention data, the model also predicts in-car driver attention data of routine driving with state-of-the-art accuracy. This result not only demonstrates the performance of our model but also proves the validity and usefulness of our dataset and data collection protocol.