Incorporating Rather Than Eliminating: Achieving Fairness for Skin Disease Diagnosis Through Group-Specific Expert

TL;DR

This paper introduces FairMoE, a novel framework that improves fairness in skin disease diagnosis AI systems by incorporating sensitive attributes through group-specific experts, enhancing accuracy without sacrificing fairness.

Contribution

The paper presents FairMoE, a dynamic mixture-of-experts model that incorporates sensitive attributes to improve fairness and accuracy in skin disease diagnosis.

Findings

FairMoE outperforms traditional bias mitigation methods in accuracy.

FairMoE maintains comparable fairness metrics while improving performance.

Dynamic routing to group-specific experts effectively handles boundary cases.

Abstract

AI-based systems have achieved high accuracy in skin disease diagnostics but often exhibit biases across demographic groups, leading to inequitable healthcare outcomes and diminished patient trust. Most existing bias mitigation methods attempt to eliminate the correlation between sensitive attributes and diagnostic prediction, but those methods often degrade performance due to the lost of clinically relevant diagnostic cues. In this work, we propose an alternative approach that incorporates sensitive attributes to achieve fairness. We introduce FairMoE, a framework that employs layer-wise mixture-of-experts modules to serve as group-specific learners. Unlike traditional methods that rigidly assign data based on group labels, FairMoE dynamically routes data to the most suitable expert, making it particularly effective for handling cases near group boundaries. Experimental results show that, unlike previous fairness approaches that reduce performance, FairMoE achieves substantial accuracy improvements while preserving comparable fairness metrics.