Characterizing and modeling harms from interactions with design patterns in AI interfaces

TL;DR

This paper reviews harmful design patterns in AI interfaces, introduces the DECAI model based on control systems theory to assess impacts, and demonstrates its application through case studies on recommendation and conversational AI systems.

Contribution

It provides a comprehensive review of harmful AI interface patterns and proposes DECAI, a novel impact assessment framework based on control theory.

Findings

Identified key harmful design patterns in AI interfaces

Demonstrated DECAI's effectiveness through case studies

Highlighted the importance of impact assessment in AI interface design

Abstract

The proliferation of applications using artificial intelligence (AI) systems has led to a growing number of users interacting with these systems through sophisticated interfaces. Human-computer interaction research has long shown that interfaces shape both user behavior and user perception of technical capabilities and risks. Yet, practitioners and researchers evaluating the social and ethical risks of AI systems tend to overlook the impact of anthropomorphic, deceptive, and immersive interfaces on human-AI interactions. Here, we argue that design features of interfaces with adaptive AI systems can have cascading impacts, driven by feedback loops, which extend beyond those previously considered. We first conduct a scoping review of AI interface designs and their negative impact to extract salient themes of potentially harmful design patterns in AI interfaces. Then, we propose Design-Enhanced Control of AI systems (DECAI), a conceptual model to structure and facilitate impact assessments of AI interface designs. DECAI draws on principles from control systems theory -- a theory for the analysis and design of dynamic physical systems -- to dissect the role of the interface in human-AI systems. Through two case studies on recommendation systems and conversational language model systems, we show how DECAI can be used to evaluate AI interface designs.