Text2VLM: Adapting Text-Only Datasets to Evaluate Alignment Training in Visual Language Models

TL;DR

Text2VLM introduces a pipeline to convert text-only datasets into multimodal prompts, enabling evaluation of VLMs' robustness against prompt injection attacks and revealing vulnerabilities in current models.

Contribution

The paper presents a novel multi-stage pipeline that adapts text-only datasets into multimodal formats for evaluating VLM alignment and robustness.

Findings

Open-source VLMs are more vulnerable to prompt injection with visual inputs.

Significant performance gap between open-source and closed-source VLMs.

Text2VLM is validated through human evaluations and aligns with human expectations.

Abstract

The increasing integration of Visual Language Models (VLMs) into AI systems necessitates robust model alignment, especially when handling multimodal content that combines text and images. Existing evaluation datasets heavily lean towards text-only prompts, leaving visual vulnerabilities under evaluated. To address this gap, we propose \textbf{Text2VLM}, a novel multi-stage pipeline that adapts text-only datasets into multimodal formats, specifically designed to evaluate the resilience of VLMs against typographic prompt injection attacks. The Text2VLM pipeline identifies harmful content in the original text and converts it into a typographic image, creating a multimodal prompt for VLMs. Also, our evaluation of open-source VLMs highlights their increased susceptibility to prompt injection when visual inputs are introduced, revealing critical weaknesses in the current models' alignment. This is in addition to a significant performance gap compared to closed-source frontier models. We validate Text2VLM through human evaluations, ensuring the alignment of extracted salient concepts; text summarization and output classification align with human expectations. Text2VLM provides a scalable tool for comprehensive safety assessment, contributing to the development of more robust safety mechanisms for VLMs. By enhancing the evaluation of multimodal vulnerabilities, Text2VLM plays a role in advancing the safe deployment of VLMs in diverse, real-world applications.