FactSheets: Increasing Trust in AI Services through Supplier's Declarations of Conformity

TL;DR

This paper proposes FactSheets, standardized documents inspired by industry practices, to enhance trust in AI services by transparently communicating safety, performance, security, and provenance information.

Contribution

It introduces a comprehensive set of declaration items for AI services and demonstrates their application through fictitious examples, aiming to improve transparency and trust.

Findings

FactSheets can effectively communicate AI safety and performance details.

They serve as multi-dimensional trust-building documents for consumers.

The approach is inspired by industry standards like SDoCs.

Abstract

Accuracy is an important concern for suppliers of artificial intelligence (AI) services, but considerations beyond accuracy, such as safety (which includes fairness and explainability), security, and provenance, are also critical elements to engender consumers' trust in a service. Many industries use transparent, standardized, but often not legally required documents called supplier's declarations of conformity (SDoCs) to describe the lineage of a product along with the safety and performance testing it has undergone. SDoCs may be considered multi-dimensional fact sheets that capture and quantify various aspects of the product and its development to make it worthy of consumers' trust. Inspired by this practice, we propose FactSheets to help increase trust in AI services. We envision such documents to contain purpose, performance, safety, security, and provenance information to be completed by AI service providers for examination by consumers. We suggest a comprehensive set of declaration items tailored to AI and provide examples for two fictitious AI services in the appendix of the paper.