Efficient DNN-Powered Software with Fair Sparse Models

TL;DR

This paper introduces Ballot, a novel pruning framework that enhances fairness in DNN models by addressing fairness issues in existing pruning methods, achieving significant fairness improvements across multiple datasets and models.

Contribution

Ballot employs conflict-detection-based subnetwork selection and refined training to improve fairness in model pruning, addressing limitations of existing methods like LTH.

Findings

Ballot improves fairness by up to 38% over baselines.

It outperforms Magnitude Pruning, LTH, SafeCompress, and FairScratch.

Evaluated on five datasets and three models.

Abstract

With the emergence of the Software 3.0 era, there is a growing trend of compressing and integrating large models into software systems, with significant societal implications. Regrettably, in numerous instances, model compression techniques impact the fairness performance of these models and thus the ethical behavior of DNN-powered software. One of the most notable example is the Lottery Ticket Hypothesis (LTH), a prevailing model pruning approach. This paper demonstrates that fairness issue of LTHbased pruning arises from both its subnetwork selection and training procedures, highlighting the inadequacy of existing remedies. To address this, we propose a novel pruning framework, Ballot, which employs a novel conflict-detection-based subnetwork selection to find accurate and fair subnetworks, coupled with a refined training process to attain a high-performance model, thereby improving the fairness of DNN-powered software. By means of this procedure, Ballot improves the fairness of pruning by 38.00%, 33.91%, 17.96%, and 35.82% compared to state-of-the-art baselines, namely Magnitude Pruning, Standard LTH, SafeCompress, and FairScratch respectively, based on our evaluation of five popular datasets and three widely used models. Our code is available at https://anonymous.4open.science/r/Ballot-506E.