Cause-Effect Driven Optimization for Robust Medical Visual Question Answering with Language Biases

TL;DR

This paper introduces CEDO, a comprehensive framework that reduces language biases in medical visual question answering by employing modality-specific optimization, gradient-based synergy, and adaptive loss rescaling, leading to more robust reasoning.

Contribution

The paper proposes a novel Cause-Effect Driven Optimization framework with three mechanisms to mitigate language biases from causal and effectual perspectives in Med-VQA models.

Findings

CEDO outperforms state-of-the-art methods on multiple benchmarks.

The framework effectively reduces shortcut and dataset imbalance biases.

Extensive experiments validate the robustness of CEDO across various datasets.

Abstract

Existing Medical Visual Question Answering (Med-VQA) models often suffer from language biases, where spurious correlations between question types and answer categories are inadvertently established. To address these issues, we propose a novel Cause-Effect Driven Optimization framework called CEDO, that incorporates three well-established mechanisms, i.e., Modality-driven Heterogeneous Optimization (MHO), Gradient-guided Modality Synergy (GMS), and Distribution-adapted Loss Rescaling (DLR), for comprehensively mitigating language biases from both causal and effectual perspectives. Specifically, MHO employs adaptive learning rates for specific modalities to achieve heterogeneous optimization, thus enhancing robust reasoning capabilities. Additionally, GMS leverages the Pareto optimization method to foster synergistic interactions between modalities and enforce gradient orthogonality to eliminate bias updates, thereby mitigating language biases from the effect side, i.e., shortcut bias. Furthermore, DLR is designed to assign adaptive weights to individual losses to ensure balanced learning across all answer categories, effectively alleviating language biases from the cause side, i.e., imbalance biases within datasets. Extensive experiments on multiple traditional and bias-sensitive benchmarks consistently demonstrate the robustness of CEDO over state-of-the-art competitors.