Is Your Text-to-Image Model Robust to Caption Noise?

TL;DR

This paper investigates how caption hallucinations from Vision Language Models affect text-to-image generation, revealing the importance of caption quality and proposing a confidence-score-based method to improve model robustness.

Contribution

The study systematically analyzes caption hallucination effects on T2I models and introduces a novel approach using VLM confidence scores to mitigate caption noise.

Findings

Caption quality disparities impact T2I outputs.

VLM confidence scores reliably detect caption noise.

Subtle caption fidelity changes significantly affect learned representations.

Abstract

In text-to-image (T2I) generation, a prevalent training technique involves utilizing Vision Language Models (VLMs) for image re-captioning. Even though VLMs are known to exhibit hallucination, generating descriptive content that deviates from the visual reality, the ramifications of such caption hallucinations on T2I generation performance remain under-explored. Through our empirical investigation, we first establish a comprehensive dataset comprising VLM-generated captions, and then systematically analyze how caption hallucination influences generation outcomes. Our findings reveal that (1) the disparities in caption quality persistently impact model outputs during fine-tuning. (2) VLMs confidence scores serve as reliable indicators for detecting and characterizing noise-related patterns in the data distribution. (3) even subtle variations in caption fidelity have significant effects on the quality of learned representations. These findings collectively emphasize the profound impact of caption quality on model performance and highlight the need for more sophisticated robust training algorithm in T2I. In response to these observations, we propose a approach leveraging VLM confidence score to mitigate caption noise, thereby enhancing the robustness of T2I models against hallucination in caption.