Open Problems in Computer Vision for Wilderness SAR and The Search for Patricia Wu-Murad

TL;DR

This paper examines the challenges of applying computer vision models to drone imagery in wilderness search and rescue, highlighting the gap between dataset performance and real-world effectiveness, and proposing future research directions.

Contribution

It identifies key limitations of current models in real WSAR scenarios and suggests three critical areas for future research to improve practical applicability.

Findings

EfficientDET performs similarly to state-of-the-art on datasets but struggles in real-world WSAR.

Unsupervised RX spectral classifier was selected as suitable for wilderness imagery.

Future research should focus on realistic datasets, adaptable models, and aligned performance metrics.

Abstract

This paper details the challenges in applying two computer vision systems, an EfficientDET supervised learning model and the unsupervised RX spectral classifier, to 98.9 GB of drone imagery from the Wu-Murad wilderness search and rescue (WSAR) effort in Japan and identifies 3 directions for future research. There have been at least 19 proposed approaches and 3 datasets aimed at locating missing persons in drone imagery, but only 3 approaches (2 unsupervised and 1 of an unknown structure) are referenced in the literature as having been used in an actual WSAR operation. Of these proposed approaches, the EfficientDET architecture and the unsupervised spectral RX classifier were selected as the most appropriate for this setting. The EfficientDET model was applied to the HERIDAL dataset and despite achieving performance that is statistically equivalent to the state-of-the-art, the model fails to translate to the real world in terms of false positives (e.g., identifying tree limbs and rocks as people), and false negatives (e.g., failing to identify members of the search team). The poor results in practice for algorithms that showed good results on datasets suggest 3 areas of future research: more realistic datasets for wilderness SAR, computer vision models that are capable of seamlessly handling the variety of imagery that can be collected during actual WSAR operations, and better alignment on performance measures.