Achieving Fairness Without Harm via Selective Demographic Experts

TL;DR

This paper introduces a novel method for fairness in machine learning that learns separate representations for demographic groups and selectively applies group-specific classifiers, ensuring fairness without sacrificing accuracy, especially in sensitive medical applications.

Contribution

The paper proposes a fairness-without-harm approach using group-specific representations and a no-harm constrained selection mechanism, advancing fairness techniques without degrading performance.

Findings

Effective in achieving fairness without harm across multiple medical datasets

Maintains high predictive accuracy while ensuring fairness

Outperforms existing bias mitigation methods in real-world scenarios

Abstract

As machine learning systems become increasingly integrated into human-centered domains such as healthcare, ensuring fairness while maintaining high predictive performance is critical. Existing bias mitigation techniques often impose a trade-off between fairness and accuracy, inadvertently degrading performance for certain demographic groups. In high-stakes domains like clinical diagnosis, such trade-offs are ethically and practically unacceptable. In this study, we propose a fairness-without-harm approach by learning distinct representations for different demographic groups and selectively applying demographic experts consisting of group-specific representations and personalized classifiers through a no-harm constrained selection. We evaluate our approach on three real-world medical datasets -- covering eye disease, skin cancer, and X-ray diagnosis -- as well as two face datasets. Extensive empirical results demonstrate the effectiveness of our approach in achieving fairness without harm.