Through the Looking-Glass: Transparency Implications and Challenges in Enterprise AI Knowledge Systems

TL;DR

This paper analyzes transparency in enterprise AI knowledge systems, emphasizing their sociotechnical nature and the importance of multiple transparency dimensions to mitigate harms and improve understanding.

Contribution

It introduces the looking-glass metaphor to conceptualize transparency as reflection and distortion, expanding understanding of transparency challenges in sociotechnical AI systems.

Findings

Identifies three key transparency dimensions: system, procedural, and outcome transparency.

Highlights the sociotechnical gap hindering transparency implementation.

Proposes future research directions in CSCW for better transparency practices.

Abstract

Knowledge can't be disentangled from people. As AI knowledge systems mine vast volumes of work-related data, the knowledge that's being extracted and surfaced is intrinsically linked to the people who create and use it. When predictive algorithms that learn from data are used to link knowledge and people, inaccuracies in knowledge extraction and surfacing can lead to disproportionate harms, influencing how individuals see each other and how they see themselves at work. In this paper, we present a reflective analysis of transparency requirements and impacts in this type of systems. We conduct a multidisciplinary literature review to understand the impacts of transparency in workplace settings, introducing the looking-glass metaphor to conceptualize AI knowledge systems as systems that reflect and distort, expanding our view on transparency requirements, implications and challenges. We formulate transparency as a key mediator in shaping different ways of seeing, including seeing into the system, which unveils its capabilities, limitations and behavior, and seeing through the system, which shapes workers' perceptions of their own contributions and others within the organization. Recognizing the sociotechnical nature of these systems, we identify three transparency dimensions necessary to realize the value of AI knowledge systems, namely system transparency, procedural transparency and transparency of outcomes. We discuss key challenges hindering the implementation of these forms of transparency, bringing to light the wider sociotechnical gap and highlighting directions for future Computer-supported Cooperative Work (CSCW) research.