Deep Clustering based Fair Outlier Detection

TL;DR

This paper introduces DCFOD, a deep clustering method that enhances fair outlier detection by learning subgroup-invariant representations and balancing utility with fairness, outperforming existing algorithms.

Contribution

The paper presents a novel deep clustering approach with adversarial training and weighted representation learning to achieve fair and effective outlier detection.

Findings

Consistently outperforms 17 algorithms on 8 datasets.

Achieves superior fairness and detection validity.

Effectively mitigates bias while detecting outliers.

Abstract

In this paper, we focus on the fairness issues regarding unsupervised outlier detection. Traditional algorithms, without a specific design for algorithmic fairness, could implicitly encode and propagate statistical bias in data and raise societal concerns. To correct such unfairness and deliver a fair set of potential outlier candidates, we propose Deep Clustering based Fair Outlier Detection (DCFOD) that learns a good representation for utility maximization while enforcing the learnable representation to be subgroup-invariant on the sensitive attribute. Considering the coupled and reciprocal nature between clustering and outlier detection, we leverage deep clustering to discover the intrinsic cluster structure and out-of-structure instances. Meanwhile, an adversarial training erases the sensitive pattern for instances for fairness adaptation. Technically, we propose an instance-level weighted representation learning strategy to enhance the joint deep clustering and outlier detection, where the dynamic weight module re-emphasizes contributions of likely-inliers while mitigating the negative impact from outliers. Demonstrated by experiments on eight datasets comparing to 17 outlier detection algorithms, our DCFOD method consistently achieves superior performance on both the outlier detection validity and two types of fairness notions in outlier detection.