Cross-Modal Safety Alignment: Is textual unlearning all you need?

TL;DR

This paper investigates whether unlearning in the textual domain alone can effectively improve safety in vision-language models, demonstrating significant reduction in attack success rates with minimal utility loss.

Contribution

The study shows that textual unlearning in vision-language models effectively reduces attack success rates, offering a simpler alternative to multi-modal safety training.

Findings

Textual unlearning reduces attack success rate to below 8%.

Unlearning with multi-modal data offers no benefits and increases computational costs.

Safety can be improved without extensive multi-modal dataset collection.

Abstract

Recent studies reveal that integrating new modalities into Large Language Models (LLMs), such as Vision-Language Models (VLMs), creates a new attack surface that bypasses existing safety training techniques like Supervised Fine-tuning (SFT) and Reinforcement Learning with Human Feedback (RLHF). While further SFT and RLHF-based safety training can be conducted in multi-modal settings, collecting multi-modal training datasets poses a significant challenge. Inspired by the structural design of recent multi-modal models, where, regardless of the combination of input modalities, all inputs are ultimately fused into the language space, we aim to explore whether unlearning solely in the textual domain can be effective for cross-modality safety alignment. Our evaluation across six datasets empirically demonstrates the transferability -- textual unlearning in VLMs significantly reduces the Attack Success Rate (ASR) to less than 8\% and in some cases, even as low as nearly 2\% for both text-based and vision-text-based attacks, alongside preserving the utility. Moreover, our experiments show that unlearning with a multi-modal dataset offers no potential benefits but incurs significantly increased computational demands, possibly up to 6 times higher.