Investigating and Designing for Trust in AI-powered Code Generation Tools

TL;DR

This paper explores how developers trust AI code generation tools, identifying challenges and proposing design features like performance communication, configurability, and transparency to foster appropriate trust levels.

Contribution

It provides a qualitative analysis of trust issues and introduces design concepts to improve trust calibration in AI-powered coding tools.

Findings

Developers face challenges in setting expectations, configuring, and validating AI suggestions.

Design concepts like performance indicators and configurability can support trust-building.

Feedback highlights potential risks and benefits of proposed trust-supporting features.

Abstract

As AI-powered code generation tools such as GitHub Copilot become popular, it is crucial to understand software developers' trust in AI tools -- a key factor for tool adoption and responsible usage. However, we know little about how developers build trust with AI, nor do we understand how to design the interface of generative AI systems to facilitate their appropriate levels of trust. In this paper, we describe findings from a two-stage qualitative investigation. We first interviewed 17 developers to contextualize their notions of trust and understand their challenges in building appropriate trust in AI code generation tools. We surfaced three main challenges -- including building appropriate expectations, configuring AI tools, and validating AI suggestions. To address these challenges, we conducted a design probe study in the second stage to explore design concepts that support developers' trust-building process by 1) communicating AI performance to help users set proper expectations, 2) allowing users to configure AI by setting and adjusting preferences, and 3) offering indicators of model mechanism to support evaluation of AI suggestions. We gathered developers' feedback on how these design concepts can help them build appropriate trust in AI-powered code generation tools, as well as potential risks in design. These findings inform our proposed design recommendations on how to design for trust in AI-powered code generation tools.