MEDIRL: Predicting the Visual Attention of Drivers via Maximum Entropy Deep Inverse Reinforcement Learning

TL;DR

This paper introduces MEDIRL, a novel deep inverse reinforcement learning approach that predicts driver visual attention in accident-prone scenarios, outperforming existing models on multiple benchmarks.

Contribution

The paper presents MEDIRL, a new maximum entropy deep inverse reinforcement learning framework, and introduces the EyeCar dataset for driver attention in risky situations.

Findings

MEDIRL outperforms existing attention prediction models.

State-of-the-art results on DR(eye)VE, BDD-A, DADA-2000, and EyeCar datasets.

Extensive ablation studies validate the model's effectiveness.

Abstract

Inspired by human visual attention, we propose a novel inverse reinforcement learning formulation using Maximum Entropy Deep Inverse Reinforcement Learning (MEDIRL) for predicting the visual attention of drivers in accident-prone situations. MEDIRL predicts fixation locations that lead to maximal rewards by learning a task-sensitive reward function from eye fixation patterns recorded from attentive drivers. Additionally, we introduce EyeCar, a new driver attention dataset in accident-prone situations. We conduct comprehensive experiments to evaluate our proposed model on three common benchmarks: (DR(eye)VE, BDD-A, DADA-2000), and our EyeCar dataset. Results indicate that MEDIRL outperforms existing models for predicting attention and achieves state-of-the-art performance. We present extensive ablation studies to provide more insights into different features of our proposed model.