Differentiating Objects by Motion: Joint Detection and Tracking of Small Flying Objects

TL;DR

This paper introduces a neural model that jointly detects and tracks small flying objects by leveraging motion cues across multiple frames, significantly improving detection accuracy over existing methods.

Contribution

The paper proposes the Recurrent Correlational Network, a novel end-to-end trainable model that jointly learns detection and tracking of small objects using multi-frame motion information.

Findings

Improved detection performance on small flying objects.

Achieved results comparable to state-of-the-art trackers.

Enhanced detection over single-frame detectors.

Abstract

While generic object detection has achieved large improvements with rich feature hierarchies from deep nets, detecting small objects with poor visual cues remains challenging. Motion cues from multiple frames may be more informative for detecting such hard-to-distinguish objects in each frame. However, how to encode discriminative motion patterns, such as deformations and pose changes that characterize objects, has remained an open question. To learn them and thereby realize small object detection, we present a neural model called the Recurrent Correlational Network, where detection and tracking are jointly performed over a multi-frame representation learned through a single, trainable, and end-to-end network. A convolutional long short-term memory network is utilized for learning informative appearance change for detection, while learned representation is shared in tracking for enhancing its performance. In experiments with datasets containing images of scenes with small flying objects, such as birds and unmanned aerial vehicles, the proposed method yielded consistent improvements in detection performance over deep single-frame detectors and existing motion-based detectors. Furthermore, our network performs as well as state-of-the-art generic object trackers when it was evaluated as a tracker on the bird dataset.