VISTA: A Visual and Textual Attention Dataset for Interpreting Multimodal Models

TL;DR

This paper introduces a new dataset that aligns human visual attention with image-text pairs to analyze and interpret the decision-making processes of vision-language models, enhancing transparency and trust.

Contribution

It provides an image-text aligned human visual attention dataset and a methodology to compare model heatmaps with human attention for better interpretability.

Findings

Models' attention aligns variably with human attention

Analysis reveals strengths and weaknesses in model interpretability

Dataset enables systematic evaluation of visual-text associations

Abstract

The recent developments in deep learning led to the integration of natural language processing (NLP) with computer vision, resulting in powerful integrated Vision and Language Models (VLMs). Despite their remarkable capabilities, these models are frequently regarded as black boxes within the machine learning research community. This raises a critical question: which parts of an image correspond to specific segments of text, and how can we decipher these associations? Understanding these connections is essential for enhancing model transparency, interpretability, and trustworthiness. To answer this question, we present an image-text aligned human visual attention dataset that maps specific associations between image regions and corresponding text segments. We then compare the internal heatmaps generated by VL models with this dataset, allowing us to analyze and better understand the model's decision-making process. This approach aims to enhance model transparency, interpretability, and trustworthiness by providing insights into how these models align visual and linguistic information. We conducted a comprehensive study on text-guided visual saliency detection in these VL models. This study aims to understand how different models prioritize and focus on specific visual elements in response to corresponding text segments, providing deeper insights into their internal mechanisms and improving our ability to interpret their outputs.