The Rise of the AI Co-Pilot: Lessons for Design from Aviation and Beyond

TL;DR

This paper advocates for designing AI systems as collaborative co-pilots that enhance human oversight and engagement, drawing lessons from aviation to prevent over-reliance and skill erosion in AI-human partnerships.

Contribution

It introduces a design framework emphasizing active human control and oversight in AI systems, inspired by aviation safety principles, to foster effective human-AI collaboration.

Findings

Highlighting risks of automation over-reliance

Proposing design principles for human-centered AI interaction

Emphasizing the importance of human oversight in AI systems

Abstract

The fast pace of advances in AI promises to revolutionize various aspects of knowledge work, extending its influence to daily life and professional fields alike. We advocate for a paradigm where AI is seen as a collaborative co-pilot, working under human guidance rather than as a mere tool. Drawing from relevant research and literature in the disciplines of Human-Computer Interaction and Human Factors Engineering, we highlight the criticality of maintaining human oversight in AI interactions. Reflecting on lessons from aviation, we address the dangers of over-relying on automation, such as diminished human vigilance and skill erosion. Our paper proposes a design approach that emphasizes active human engagement, control, and skill enhancement in the AI partnership, aiming to foster a harmonious, effective, and empowering human-AI relationship. We particularly call out the critical need to design AI interaction capabilities and software applications to enable and celebrate the primacy of human agency. This calls for designs for human-AI partnership that cede ultimate control and responsibility to the human user as pilot, with the AI co-pilot acting in a well-defined supporting role.