DriveBLIP2: Attention-Guided Explanation Generation for Complex Driving Scenarios

TL;DR

DriveBLIP2 enhances explanation generation for complex driving scenarios by integrating attention mechanisms that highlight key objects, improving interpretability and decision understanding in autonomous driving contexts.

Contribution

The paper introduces DriveBLIP2, a novel framework that incorporates an attention map generator into vision-language models to improve explanation relevance in complex driving environments.

Findings

Significant improvements in explanation quality metrics on DRAMA dataset

Effective highlighting of key objects improves model interpretability

Enhanced real-time understanding of driving decisions

Abstract

This paper introduces a new framework, DriveBLIP2, built upon the BLIP2-OPT architecture, to generate accurate and contextually relevant explanations for emerging driving scenarios. While existing vision-language models perform well in general tasks, they encounter difficulties in understanding complex, multi-object environments, particularly in real-time applications such as autonomous driving, where the rapid identification of key objects is crucial. To address this limitation, an Attention Map Generator is proposed to highlight significant objects relevant to driving decisions within critical video frames. By directing the model's focus to these key regions, the generated attention map helps produce clear and relevant explanations, enabling drivers to better understand the vehicle's decision-making process in critical situations. Evaluations on the DRAMA dataset reveal significant improvements in explanation quality, as indicated by higher BLEU, ROUGE, CIDEr, and SPICE scores compared to baseline models. These findings underscore the potential of targeted attention mechanisms in vision-language models for enhancing explainability in real-time autonomous driving.