Towards Fairness Certification in Artificial Intelligence

TL;DR

This paper discusses establishing guidelines and certification procedures to ensure fairness in AI systems, especially in sensitive societal applications, by defining operational criteria and assessment methods.

Contribution

It introduces a joint framework for fairness certification in AI, combining standards from measurement and machine learning expertise to guide deployment and monitoring.

Findings

Proposes operational steps for AI fairness certification.

Defines criteria for AI systems before deployment.

Outlines conformity assessment procedures for ongoing monitoring.

Abstract

Thanks to the great progress of machine learning in the last years, several Artificial Intelligence (AI) techniques have been increasingly moving from the controlled research laboratory settings to our everyday life. AI is clearly supportive in many decision-making scenarios, but when it comes to sensitive areas such as health care, hiring policies, education, banking or justice, with major impact on individuals and society, it becomes crucial to establish guidelines on how to design, develop, deploy and monitor this technology. Indeed the decision rules elaborated by machine learning models are data-driven and there are multiple ways in which discriminatory biases can seep into data. Algorithms trained on those data incur the risk of amplifying prejudices and societal stereotypes by over associating protected attributes such as gender, ethnicity or disabilities with the prediction task. Starting from the extensive experience of the National Metrology Institute on measurement standards and certification roadmaps, and of Politecnico di Torino on machine learning as well as methods for domain bias evaluation and mastering, we propose a first joint effort to define the operational steps needed for AI fairness certification. Specifically we will overview the criteria that should be met by an AI system before coming into official service and the conformity assessment procedures useful to monitor its functioning for fair decisions.