# Assessment of Faster R-CNN in Man-Machine collaborative search

## TL;DR

This study evaluates how a Faster R-CNN-based deep learning system influences human performance in a detailed visual search task, revealing benefits for high-sensitivity observers and limitations for low-sensitivity ones.

## Contribution

It provides empirical insights into the interaction between deep learning expert systems and human visual search performance, highlighting conditions of collaboration and competition.

## Key findings

- DL system reduces false alarms across observer groups
- High sensitivity humans outperform DL system alone, low sensitivity do not
- DL aids fixation at target by third fixation

## Abstract

With the advent of modern expert systems driven by deep learning that supplement human experts (e.g. radiologists, dermatologists, surveillance scanners), we analyze how and when do such expert systems enhance human performance in a fine-grained small target visual search task. We set up a 2 session factorial experimental design in which humans visually search for a target with and without a Deep Learning (DL) expert system. We evaluate human changes of target detection performance and eye-movements in the presence of the DL system. We find that performance improvements with the DL system (computed via a Faster R-CNN with a VGG16) interacts with observer's perceptual abilities (e.g., sensitivity). The main results include: 1) The DL system reduces the False Alarm rate per Image on average across observer groups of both high/low sensitivity; 2) Only human observers with high sensitivity perform better than the DL system, while the low sensitivity group does not surpass individual DL system performance, even when aided with the DL system itself; 3) Increases in number of trials and decrease in viewing time were mainly driven by the DL system only for the low sensitivity group. 4) The DL system aids the human observer to fixate at a target by the 3rd fixation. These results provide insights of the benefits and limitations of deep learning systems that are collaborative or competitive with humans.