The Fairness-Quality Trade-off in Clustering

TL;DR

This paper introduces algorithms to explore the full trade-off between fairness and quality in clustering, addressing a gap in understanding how to balance these objectives effectively.

Contribution

It presents novel algorithms for computing the entire Pareto front of fairness and quality in clustering, including a polynomial-time method for fixed centers and a general complexity analysis.

Findings

The Pareto front can be exponential in size, making exact computation computationally hard.

A polynomial-time algorithm exists for fixed cluster centers and certain fairness objectives.

Computational complexity results show the problem is NP-hard in general.

Abstract

Fairness in clustering has been considered extensively in the past; however, the trade-off between the two objectives -- e.g., can we sacrifice just a little in the quality of the clustering to significantly increase fairness, or vice-versa? -- has rarely been addressed. We introduce novel algorithms for tracing the complete trade-off curve, or Pareto front, between quality and fairness in clustering problems; that is, computing all clusterings that are not dominated in both objectives by other clusterings. Unlike previous work that deals with specific objectives for quality and fairness, we deal with all objectives for fairness and quality in two general classes encompassing most of the special cases addressed in previous work. Our algorithm must take exponential time in the worst case as the Pareto front itself can be exponential. Even when the Pareto front is polynomial, our algorithm may take exponential time, and we prove that this is inevitable unless P = NP. However, we also present a new polynomial-time algorithm for computing the entire Pareto front when the cluster centers are fixed, and for perhaps the most natural fairness objective: minimizing the sum, over all clusters, of the imbalance between the two groups in each cluster.