Memory Enhanced Global-Local Aggregation for Video Object Detection

TL;DR

This paper introduces the MEGA network, which combines global semantic and local localization information with a novel memory module to improve video object detection, achieving state-of-the-art results.

Contribution

The paper proposes a novel memory enhanced global-local aggregation network that fully leverages both global and local cues for improved video object detection.

Findings

Achieves state-of-the-art performance on ImageNet VID dataset.

Introduces a Long Range Memory module for better feature access.

Effectively combines global and local information for detection.

Abstract

How do humans recognize an object in a piece of video? Due to the deteriorated quality of single frame, it may be hard for people to identify an occluded object in this frame by just utilizing information within one image. We argue that there are two important cues for humans to recognize objects in videos: the global semantic information and the local localization information. Recently, plenty of methods adopt the self-attention mechanisms to enhance the features in key frame with either global semantic information or local localization information. In this paper we introduce memory enhanced global-local aggregation (MEGA) network, which is among the first trials that takes full consideration of both global and local information. Furthermore, empowered by a novel and carefully-designed Long Range Memory (LRM) module, our proposed MEGA could enable the key frame to get access to much more content than any previous methods. Enhanced by these two sources of information, our method achieves state-of-the-art performance on ImageNet VID dataset. Code is available at \url{https://github.com/Scalsol/mega.pytorch}.