Fair Sparse Regression with Clustering: An Invex Relaxation for a Combinatorial Problem

TL;DR

This paper introduces a novel invex relaxation approach to fair sparse regression with hidden biases, enabling exact recovery of parameters and clustering in a combinatorial setting with theoretical guarantees.

Contribution

It proposes the first invex relaxation for a combinatorial problem involving fair sparse regression and clustering with unknown binary labels.

Findings

Exact recovery of the true parameter vector.

Simultaneous clustering and debiasing without performance loss.

Logarithmic sample complexity in problem dimension.

Abstract

In this paper, we study the problem of fair sparse regression on a biased dataset where bias depends upon a hidden binary attribute. The presence of a hidden attribute adds an extra layer of complexity to the problem by combining sparse regression and clustering with unknown binary labels. The corresponding optimization problem is combinatorial, but we propose a novel relaxation of it as an \emph{invex} optimization problem. To the best of our knowledge, this is the first invex relaxation for a combinatorial problem. We show that the inclusion of the debiasing/fairness constraint in our model has no adverse effect on the performance. Rather, it enables the recovery of the hidden attribute. The support of our recovered regression parameter vector matches exactly with the true parameter vector. Moreover, we simultaneously solve the clustering problem by recovering the exact value of the hidden attribute for each sample. Our method uses carefully constructed primal dual witnesses to provide theoretical guarantees for the combinatorial problem. To that end, we show that the sample complexity of our method is logarithmic in terms of the dimension of the regression parameter vector.