Mapping User Trust in Vision Language Models: Research Landscape, Challenges, and Prospects

TL;DR

This survey explores how users develop trust in Vision Language Models, analyzing current research, challenges, and future directions to ensure reliable and transparent AI-human interactions.

Contribution

It provides a comprehensive taxonomy of trust dynamics in VLMs and offers preliminary requirements for future trust-related research in this domain.

Findings

Identifies key factors influencing user trust in VLMs

Highlights gaps in current trust research and understanding

Proposes a multidisciplinary framework for studying trust in VLMs

Abstract

The rapid adoption of Vision Language Models (VLMs), pre-trained on large image-text and video-text datasets, calls for protecting and informing users about when to trust these systems. This survey reviews studies on trust dynamics in user-VLM interactions, through a multi-disciplinary taxonomy encompassing different cognitive science capabilities, collaboration modes, and agent behaviours. Literature insights and findings from a workshop with prospective VLM users inform preliminary requirements for future VLM trust studies.