debiaSAE: Benchmarking and Mitigating Vision-Language Model Bias

TL;DR

This paper evaluates demographic biases in vision-language models, identifies limitations of existing benchmarks, and introduces a new dataset and a Sparse Autoencoder-based debiasing method that improves fairness.

Contribution

It provides a rigorous bias evaluation dataset and a novel, interpretable debiasing approach for vision-language models, addressing current benchmark shortcomings.

Findings

Portrait datasets are effective for bias detection.

Scene-based datasets are less useful due to model guessing.

The proposed debiasing method improves fairness by 5-15 points.

Abstract

As Vision Language Models (VLMs) gain widespread use, their fairness remains under-explored. In this paper, we analyze demographic biases across five models and six datasets. We find that portrait datasets like UTKFace and CelebA are the best tools for bias detection, finding gaps in performance and fairness for both LLaVa and CLIP models. Scene-based datasets like PATA and VLStereoSet fail to be useful benchmarks for bias due to their text prompts allowing the model to guess the answer without a picture. As for pronoun-based datasets like VisoGender, we receive mixed signals as only some subsets of the data are useful in providing insights. To alleviate these two problems, we introduce a more rigorous evaluation dataset and a debiasing method based on Sparse Autoencoders to help reduce bias in models. We find that our data set generates more meaningful errors than the previous data sets. Furthermore, our debiasing method improves fairness, gaining 5-15 points in performance over the baseline. This study displays the problems with the current benchmarks for measuring demographic bias in Vision Language Models and introduces both a more effective dataset for measuring bias and a novel and interpretable debiasing method based on Sparse Autoencoders.