FactSheets: Increasing Trust in AI Services through Supplier's Declarations of Conformity
Matthew Arnold, Rachel K. E. Bellamy, Michael Hind, Stephanie Houde,, Sameep Mehta, Aleksandra Mojsilovic, Ravi Nair, Karthikeyan Natesan, Ramamurthy, Darrell Reimer, Alexandra Olteanu, David Piorkowski, Jason Tsay,, and Kush R. Varshney

TL;DR
This paper proposes FactSheets, standardized documents inspired by industry practices, to enhance trust in AI services by transparently communicating safety, performance, security, and provenance information.
Contribution
It introduces a comprehensive set of declaration items for AI services and demonstrates their application through fictitious examples, aiming to improve transparency and trust.
Findings
FactSheets can effectively communicate AI safety and performance details.
They serve as multi-dimensional trust-building documents for consumers.
The approach is inspired by industry standards like SDoCs.
Abstract
Accuracy is an important concern for suppliers of artificial intelligence (AI) services, but considerations beyond accuracy, such as safety (which includes fairness and explainability), security, and provenance, are also critical elements to engender consumers' trust in a service. Many industries use transparent, standardized, but often not legally required documents called supplier's declarations of conformity (SDoCs) to describe the lineage of a product along with the safety and performance testing it has undergone. SDoCs may be considered multi-dimensional fact sheets that capture and quantify various aspects of the product and its development to make it worthy of consumers' trust. Inspired by this practice, we propose FactSheets to help increase trust in AI services. We envision such documents to contain purpose, performance, safety, security, and provenance information to be…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
