Predicting Return-to-Manual Performance in Lower- and Higher-Degree Automation

TL;DR

This study explores how factors like workload, fatigue, and trust in automation affect a person's ability to take control when automated systems fail, in tasks like air traffic control.

Contribution

The study identifies how within-person variability in operator states predicts return-to-manual performance across different degrees of automation.

Findings

Participants using lower-degree automation resolved conflicts faster than those using higher-degree automation.

Higher workload and fatigue were linked to poorer RTM accuracy, regardless of automation level.

Higher trust in automation was associated with better RTM performance.

Abstract

To examine operator state variables (workload, fatigue, trust in automation, task engagement) that potentially predict return-to-manual (RTM) performance after automation fails to complete a task action.

Limited research has examined the extent to which within-person variability in operator states predicts RTM performance, a prerequisite to adapting work systems based on expected performance degradation/operator strain. We examine whether operator states differentially predict RTM performance as a function of degree of automation (DOA).

Participants completed a simulated air traffic control task. Conflict detection was assisted by either a higher- or lower-DOA. When automation failed to resolve a conflict, participants needed to prevent that conflict (i.e., RTM). Participants’ self-reported workload, fatigue, trust in automation, and task engagement were periodically measured.

Participants using lower DOA were faster to resolve conflicts (RTM RT) missed by automation than those using higher DOA. DOA did not moderate the relationship between operator states and RTM performance. Collapsed across DOA, increased workload (relative to participants’ own average) and increased fatigue (relative to sample average, or relative to own average) led to the resolution of fewer conflicts missed by automation (poorer RTM accuracy). Participants with higher trust (relative to own average) had higher RTM accuracy.

Variation in operator state measures of workload, fatigue, and trust can predict RTM performance. However, given some identified inconsistency in which states are predictive across studies, further research is needed.

Adaptive work systems could be designed to respond to vulnerable operator states to minimise RTM performance decrements.

Figures