ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Time

TL;DR

This paper introduces ETA, a two-phase inference-time framework for improving the safety of vision-language models by evaluating and aligning their outputs, significantly reducing unsafe responses and enhancing helpfulness.

Contribution

The paper presents a novel inference-time safety alignment method for VLMs that evaluates inputs and responses before aligning unsafe behaviors, improving safety and usefulness without extensive resources.

Findings

Reduces unsafe rate by 87.5% in cross-modality attacks

Achieves 96.6% win-ties in GPT-4 helpfulness evaluation

Outperforms baseline safety methods in experiments

Abstract

Vision Language Models (VLMs) have become essential backbones for multimodal intelligence, yet significant safety challenges limit their real-world application. While textual inputs are often effectively safeguarded, adversarial visual inputs can easily bypass VLM defense mechanisms. Existing defense methods are either resource-intensive, requiring substantial data and compute, or fail to simultaneously ensure safety and usefulness in responses. To address these limitations, we propose a novel two-phase inference-time alignment framework, Evaluating Then Aligning (ETA): 1) Evaluating input visual contents and output responses to establish a robust safety awareness in multimodal settings, and 2) Aligning unsafe behaviors at both shallow and deep levels by conditioning the VLMs' generative distribution with an interference prefix and performing sentence-level best-of-N to search the most harmless and helpful generation paths. Extensive experiments show that ETA outperforms baseline methods in terms of harmlessness, helpfulness, and efficiency, reducing the unsafe rate by 87.5% in cross-modality attacks and achieving 96.6% win-ties in GPT-4 helpfulness evaluation. The code is publicly available at https://github.com/DripNowhy/ETA.