Quantifying Cross-Modality Memorization in Vision-Language Models

TL;DR

This paper systematically investigates cross-modality memorization in vision-language models, revealing transferability of facts between modalities and identifying gaps, with implications for improving multimodal learning robustness.

Contribution

It introduces a synthetic dataset for controlled experiments and analyzes how knowledge transfers across modalities in vision-language models, highlighting existing gaps and proposing mitigation methods.

Findings

Facts learned in one modality transfer to the other.

Significant gap exists between source and target modality recall.

Transferability gap persists across various scenarios.

Abstract

Understanding what and how neural networks memorize during training is crucial, both from the perspective of unintentional memorization of potentially sensitive information and from the standpoint of effective knowledge acquisition for real-world, knowledge-intensive tasks. While previous studies primarily investigate memorization within a single modality, such as text memorization in large language models or image memorization in diffusion models, unified multimodal models are becoming increasingly prevalent in practical applications. In this work, we focus on the unique characteristics of cross-modality memorization and conduct a systematic study centered on vision-language models. To facilitate controlled experiments, we first introduce a synthetic persona dataset comprising diverse synthetic person images and textual descriptions. We quantify factual knowledge memorization and cross-modal transferability by training models on a single modality and evaluating their performance in the other. Our results reveal that facts learned in one modality transfer to the other, but a significant gap exists between recalling information in the source and target modalities. Furthermore, we observe that this gap exists across various scenarios, including more capable models, machine unlearning, and the multi-hop case. At the end, we propose a baseline method to mitigate this challenge. We hope our study can inspire future research on developing more robust multimodal learning techniques to enhance cross-modal transferability.