Beyond Recommendations: From Backward to Forward AI Support of Pilots' Decision-Making Process

TL;DR

This study compares recommendation-centric AI support with continuous, forward-support paradigms in aviation, showing that the latter enhances pilots' decision-making speed and flexibility, suggesting new directions for AI design in high-stakes environments.

Contribution

It empirically evaluates an alternative to recommendation-based AI support, demonstrating benefits of continuous, forward support in pilot decision-making during diversions.

Findings

Continuous support enables faster decision-making.

Forward support encourages broader situational awareness.

Design should focus on quick information gathering rather than recommendations.

Abstract

AI is anticipated to enhance human decision-making in high-stakes domains like aviation, but adoption is often hindered by challenges such as inappropriate reliance and poor alignment with users' decision-making. Recent research suggests that a core underlying issue is the recommendation-centric design of many AI systems, i.e., they give end-to-end recommendations and ignore the rest of the decision-making process. Alternative support paradigms are rare, and it remains unclear how the few that do exist compare to recommendation-centric support. In this work, we aimed to empirically compare recommendation-centric support to an alternative paradigm, continuous support, in the context of diversions in aviation. We conducted a mixed-methods study with 32 professional pilots in a realistic setting. To ensure the quality of our study scenarios, we conducted a focus group with four additional pilots prior to the study. We found that continuous support can support pilots' decision-making in a forward direction, allowing them to think more beyond the limits of the system and make faster decisions when combined with recommendations, though the forward support can be disrupted. Participants' statements further suggest a shift in design goal away from providing recommendations, to supporting quick information gathering. Our results show ways to design more helpful and effective AI decision support that goes beyond end-to-end recommendations.