Visual Data Diagnosis and Debiasing with Concept Graphs

TL;DR

ConBias is a framework that diagnoses and mitigates concept co-occurrence biases in visual datasets using knowledge graphs and a clique-based balancing strategy, improving model generalization.

Contribution

It introduces a novel graph-based approach for dataset diagnosis and a clique-based method for debiasing, enhancing visual dataset reliability.

Findings

Improved generalization performance on multiple datasets.

Effective identification of spurious concept co-occurrences.

Enhanced model reliability through dataset debiasing.

Abstract

The widespread success of deep learning models today is owed to the curation of extensive datasets significant in size and complexity. However, such models frequently pick up inherent biases in the data during the training process, leading to unreliable predictions. Diagnosing and debiasing datasets is thus a necessity to ensure reliable model performance. In this paper, we present ConBias, a novel framework for diagnosing and mitigating Concept co-occurrence Biases in visual datasets. ConBias represents visual datasets as knowledge graphs of concepts, enabling meticulous analysis of spurious concept co-occurrences to uncover concept imbalances across the whole dataset. Moreover, we show that by employing a novel clique-based concept balancing strategy, we can mitigate these imbalances, leading to enhanced performance on downstream tasks. Extensive experiments show that data augmentation based on a balanced concept distribution augmented by Conbias improves generalization performance across multiple datasets compared to state-of-the-art methods.