Exposing and Mitigating Calibration Biases and Demographic Unfairness in MLLM Few-Shot In-Context Learning for Medical Image Classification

TL;DR

This paper investigates calibration biases and demographic unfairness in multimodal large language models for medical image classification and introduces CALIN, a calibration method that improves fairness and accuracy across diverse demographic groups.

Contribution

The study is the first to analyze calibration biases and demographic fairness in MLLMs for medical imaging and proposes CALIN, a novel inference-time calibration technique to mitigate these biases.

Findings

CALIN improves fairness in confidence scores across demographic groups.

CALIN enhances overall prediction accuracy with minimal fairness-utility trade-off.

Experimental results on three datasets demonstrate CALIN's effectiveness in real-world scenarios.

Abstract

Multimodal large language models (MLLMs) have enormous potential to perform few-shot in-context learning in the context of medical image analysis. However, safe deployment of these models into real-world clinical practice requires an in-depth analysis of the accuracies of their predictions, and their associated calibration errors, particularly across different demographic subgroups. In this work, we present the first investigation into the calibration biases and demographic unfairness of MLLMs' predictions and confidence scores in few-shot in-context learning for medical image classification. We introduce CALIN, an inference-time calibration method designed to mitigate the associated biases. Specifically, CALIN estimates the amount of calibration needed, represented by calibration matrices, using a bi-level procedure: progressing from the population level to the subgroup level prior to inference. It then applies this estimation to calibrate the predicted confidence scores during inference. Experimental results on three medical imaging datasets: PAPILA for fundus image classification, HAM10000 for skin cancer classification, and MIMIC-CXR for chest X-ray classification demonstrate CALIN's effectiveness at ensuring fair confidence calibration in its prediction, while improving its overall prediction accuracies and exhibiting minimum fairness-utility trade-off. Our codebase can be found at https://github.com/xingbpshen/medical-calibration-fairness-mllm.