Experiences with Improving the Transparency of AI Models and Services

TL;DR

This paper explores the needs and challenges in creating transparent AI documentation through developer interviews and exercises, offering recommendations to improve AI transparency practices.

Contribution

It provides empirical insights into stakeholder needs and proposes practical recommendations for enhancing AI documentation and transparency.

Findings

Identified key challenges in AI documentation creation

Gathered stakeholder needs through interviews and exercises

Proposed flexible presentation methods for AI facts

Abstract

AI models and services are used in a growing number of highstakes areas, resulting in a need for increased transparency. Consistent with this, several proposals for higher quality and more consistent documentation of AI data, models, and systems have emerged. Little is known, however, about the needs of those who would produce or consume these new forms of documentation. Through semi-structured developer interviews, and two document creation exercises, we have assembled a clearer picture of these needs and the various challenges faced in creating accurate and useful AI documentation. Based on the observations from this work, supplemented by feedback received during multiple design explorations and stakeholder conversations, we make recommendations for easing the collection and flexible presentation of AI facts to promote transparency.