maskGRU: Tracking Small Objects in the Presence of Large Background Motions

TL;DR

maskGRU is a novel recurrent neural network framework designed to improve the detection and tracking of small objects in videos, especially amidst complex background motions, by integrating object masks into the hidden state.

Contribution

The paper introduces maskGRU, a new method that incorporates object masks into the hidden state of convGRUs, enhancing small object tracking accuracy in challenging scenarios.

Findings

maskGRU outperforms convGRU in small object tracking

Effective handling of background motions in videos

Improved stability by controlling gradient explosion

Abstract

We propose a recurrent neural network-based spatio-temporal framework named maskGRU for the detection and tracking of small objects in videos. While there have been many developments in the area of object tracking in recent years, tracking a small moving object amid other moving objects and actors (such as a ball amid moving players in sports footage) continues to be a difficult task. Existing spatio-temporal networks, such as convolutional Gated Recurrent Units (convGRUs), are difficult to train and have trouble accurately tracking small objects under such conditions. To overcome these difficulties, we developed the maskGRU framework that uses a weighted sum of the internal hidden state produced by a convGRU and a 3-channel mask of the tracked object's predicted bounding box as the hidden state to be used at the next time step of the underlying convGRU. We believe the technique of incorporating a mask into the hidden state through a weighted sum has two benefits: controlling the effect of exploding gradients and introducing an attention-like mechanism into the network by indicating where in the previous video frame the object is located. Our experiments show that maskGRU outperforms convGRU at tracking objects that are small relative to the video resolution even in the presence of other moving objects.