Optimal Fidelity Selection for Human-Supervised Search

TL;DR

This paper develops an optimal fidelity selection framework for human-supervised underwater visual search, accounting for cognitive workload and automating fidelity choices to enhance detection performance and queue management.

Contribution

It introduces a novel POMDP-based method for optimal fidelity selection considering workload, with an extension to automate task delegation for improved efficiency.

Findings

Performance improved by 26.5% without delegation

Performance improved by 50.3% with delegation

Effective workload modeling using Input-Output Hidden Markov Model

Abstract

We study optimal fidelity selection in human-supervised underwater visual search, where operator performance is affected by cognitive factors like workload and fatigue. In our experiments, participants perform two simultaneous tasks: detecting underwater mines in videos (primary) and responding to a visual cue to estimate workload (secondary). Videos arrive as a Poisson process and queue for review, with the operator choosing between normal fidelity (faster playback) and high fidelity. Rewards are based on detection accuracy, while penalties depend on queue length. Workload is modeled as a hidden state using an Input-Output Hidden Markov Model, and fidelity selection is optimized via a Partially Observable Markov Decision Process. We evaluate two setups: fidelity-only selection and a version allowing task delegation to automation to maintain queue stability. Our approach improves performance by 26.5% without delegation and 50.3% with delegation, compared to a baseline where humans manually choose their fidelity levels.