Verifiable Fairness: Privacy-preserving Computation of Fairness for Machine Learning Systems

TL;DR

This paper introduces FaaS, a secure, privacy-preserving protocol that allows for verifiable fairness assessment of machine learning models without revealing sensitive data, supporting transparency and trust.

Contribution

It presents a model-agnostic, cryptography-based protocol for privacy-preserving fairness verification that requires no trusted third parties and ensures transparency throughout the process.

Findings

FaaS supports various fairness metrics.

The protocol is secure and verifiable at each step.

Successful implementation on a large dataset.

Abstract

Fair machine learning is a thriving and vibrant research topic. In this paper, we propose Fairness as a Service (FaaS), a secure, verifiable and privacy-preserving protocol to computes and verify the fairness of any machine learning (ML) model. In the deisgn of FaaS, the data and outcomes are represented through cryptograms to ensure privacy. Also, zero knowledge proofs guarantee the well-formedness of the cryptograms and underlying data. FaaS is model--agnostic and can support various fairness metrics; hence, it can be used as a service to audit the fairness of any ML model. Our solution requires no trusted third party or private channels for the computation of the fairness metric. The security guarantees and commitments are implemented in a way that every step is securely transparent and verifiable from the start to the end of the process. The cryptograms of all input data are publicly available for everyone, e.g., auditors, social activists and experts, to verify the correctness of the process. We implemented FaaS to investigate performance and demonstrate the successful use of FaaS for a publicly available data set with thousands of entries.