Improving Human-Machine Cooperative Visual Search With Soft Highlighting

TL;DR

This paper introduces a soft highlighting technique for visual search tasks that improves human-machine cooperation by modulating saliency based on classifier confidence, outperforming traditional hard highlighting methods.

Contribution

The paper presents a novel soft highlighting approach that enhances cooperative visual search, demonstrating superior performance over hard highlighting through experiments with synthetic and natural images.

Findings

Soft highlighting outperforms hard highlighting in cooperative visual search.

The approach achieves a performance synergy exceeding traditional methods.

Experiments confirm the effectiveness of graded saliency modulation.

Abstract

Advances in machine learning have produced systems that attain human-level performance on certain visual tasks, e.g., object identification. Nonetheless, other tasks requiring visual expertise are unlikely to be entrusted to machines for some time, e.g., satellite and medical imagery analysis. We describe a human-machine cooperative approach to visual search, the aim of which is to outperform either human or machine acting alone. The traditional route to augmenting human performance with automatic classifiers is to draw boxes around regions of an image deemed likely to contain a target. Human experts typically reject this type of hard highlighting. We propose instead a soft highlighting technique in which the saliency of regions of the visual field is modulated in a graded fashion based on classifier confidence level. We report on experiments with both synthetic and natural images showing that soft highlighting achieves a performance synergy surpassing that attained by hard highlighting.