Greedy Gradient Ensemble for Robust Visual Question Answering

TL;DR

This paper introduces Greedy Gradient Ensemble, a novel de-biasing framework for Visual Question Answering that improves model robustness and visual understanding by strategically combining biased models.

Contribution

The paper proposes a new ensemble method that effectively mitigates language bias in VQA, enhancing out-of-distribution performance without extra annotations.

Findings

Achieves state-of-the-art results on VQA-CP dataset.

Effectively reduces language bias and improves visual reasoning.

Enhances model robustness to biased data distributions.

Abstract

Language bias is a critical issue in Visual Question Answering (VQA), where models often exploit dataset biases for the final decision without considering the image information. As a result, they suffer from performance drop on out-of-distribution data and inadequate visual explanation. Based on experimental analysis for existing robust VQA methods, we stress the language bias in VQA that comes from two aspects, i.e., distribution bias and shortcut bias. We further propose a new de-bias framework, Greedy Gradient Ensemble (GGE), which combines multiple biased models for unbiased base model learning. With the greedy strategy, GGE forces the biased models to over-fit the biased data distribution in priority, thus makes the base model pay more attention to examples that are hard to solve by biased models. The experiments demonstrate that our method makes better use of visual information and achieves state-of-the-art performance on diagnosing dataset VQA-CP without using extra annotations.