Fast and Efficient MMD-based Fair PCA via Optimization over Stiefel Manifold

TL;DR

This paper introduces a novel fair PCA method that minimizes MMD between class distributions, formulated as a non-convex optimization on the Stiefel manifold, and demonstrates superior performance and efficiency.

Contribution

It presents a new MMD-based fair PCA formulation solved via Riemannian optimization with theoretical guarantees and practical hyperparameter insights.

Findings

Outperforms prior methods in explained variance and fairness

Achieves faster runtime compared to existing approaches

Provides theoretical local optimality guarantees

Abstract

This paper defines fair principal component analysis (PCA) as minimizing the maximum mean discrepancy (MMD) between dimensionality-reduced conditional distributions of different protected classes. The incorporation of MMD naturally leads to an exact and tractable mathematical formulation of fairness with good statistical properties. We formulate the problem of fair PCA subject to MMD constraints as a non-convex optimization over the Stiefel manifold and solve it using the Riemannian Exact Penalty Method with Smoothing (REPMS; Liu and Boumal, 2019). Importantly, we provide local optimality guarantees and explicitly show the theoretical effect of each hyperparameter in practical settings, extending previous results. Experimental comparisons based on synthetic and UCI datasets show that our approach outperforms prior work in explained variance, fairness, and runtime.