Understanding and Mitigating Toxicity in Image-Text Pretraining Datasets: A Case Study on LLaVA

TL;DR

This paper analyzes toxicity in the LLaVA image-text dataset, identifies harmful content, and proposes mitigation strategies, resulting in a refined dataset with reduced toxic pairs to promote responsible multimodal AI development.

Contribution

It provides a detailed analysis of toxicity in a large pretraining dataset and introduces a mitigation process that removes toxic content, creating a safer dataset for multimodal models.

Findings

Identified and categorized common toxicity types in LLaVA dataset.

Removed 7,531 toxic image-text pairs from the dataset.

Provided guidelines for toxicity detection and mitigation pipelines.

Abstract

Pretraining datasets are foundational to the development of multimodal models, yet they often have inherent biases and toxic content from the web-scale corpora they are sourced from. In this paper, we investigate the prevalence of toxicity in LLaVA image-text pretraining dataset, examining how harmful content manifests in different modalities. We present a comprehensive analysis of common toxicity categories and propose targeted mitigation strategies, resulting in the creation of a refined toxicity-mitigated dataset. This dataset removes 7,531 of toxic image-text pairs in the LLaVA pre-training dataset. We offer guidelines for implementing robust toxicity detection pipelines. Our findings underscore the need to actively identify and filter toxic content - such as hate speech, explicit imagery, and targeted harassment - to build more responsible and equitable multimodal systems. The toxicity-mitigated dataset is open source and is available for further research.