ClusterNet: Detecting Small Objects in Large Scenes by Exploiting Spatio-Temporal Information

TL;DR

This paper introduces ClusterNet, a novel two-stage spatio-temporal CNN that effectively combines appearance and motion cues to detect small objects in large WAMI scenes, surpassing existing methods.

Contribution

The paper proposes ClusterNet and FoveaNet, a new two-stage CNN architecture that improves small object detection in large scenes by integrating appearance and motion information.

Findings

Outperforms state-of-the-art on WPAFB 2009 dataset by 5-16% for moving objects

Achieves nearly 50% improvement for stationary objects

First method to detect stationary objects in WAMI data

Abstract

Object detection in wide area motion imagery (WAMI) has drawn the attention of the computer vision research community for a number of years. WAMI proposes a number of unique challenges including extremely small object sizes, both sparse and densely-packed objects, and extremely large search spaces (large video frames). Nearly all state-of-the-art methods in WAMI object detection report that appearance-based classifiers fail in this challenging data and instead rely almost entirely on motion information in the form of background subtraction or frame-differencing. In this work, we experimentally verify the failure of appearance-based classifiers in WAMI, such as Faster R-CNN and a heatmap-based fully convolutional neural network (CNN), and propose a novel two-stage spatio-temporal CNN which effectively and efficiently combines both appearance and motion information to significantly surpass the state-of-the-art in WAMI object detection. To reduce the large search space, the first stage (ClusterNet) takes in a set of extremely large video frames, combines the motion and appearance information within the convolutional architecture, and proposes regions of objects of interest (ROOBI). These ROOBI can contain from one to clusters of several hundred objects due to the large video frame size and varying object density in WAMI. The second stage (FoveaNet) then estimates the centroid location of all objects in that given ROOBI simultaneously via heatmap estimation. The proposed method exceeds state-of-the-art results on the WPAFB 2009 dataset by 5-16% for moving objects and nearly 50% for stopped objects, as well as being the first proposed method in wide area motion imagery to detect completely stationary objects.