Impression Network for Video Object Detection

TL;DR

The paper introduces Impression Network, a novel video object detection method that efficiently combines features across frames using an impression mechanism, achieving high accuracy and speed simultaneously.

Contribution

It proposes a natural and efficient feature aggregation mechanism inspired by human impression, enabling long-range multi-frame fusion with minimal overhead.

Findings

Significantly improves detection accuracy on ImageNet VID

Operates at 20 fps, three times faster than previous methods

Enhances features of low-quality frames effectively

Abstract

Video object detection is more challenging compared to image object detection. Previous works proved that applying object detector frame by frame is not only slow but also inaccurate. Visual clues get weakened by defocus and motion blur, causing failure on corresponding frames. Multi-frame feature fusion methods proved effective in improving the accuracy, but they dramatically sacrifice the speed. Feature propagation based methods proved effective in improving the speed, but they sacrifice the accuracy. So is it possible to improve speed and performance simultaneously? Inspired by how human utilize impression to recognize objects from blurry frames, we propose Impression Network that embodies a natural and efficient feature aggregation mechanism. In our framework, an impression feature is established by iteratively absorbing sparsely extracted frame features. The impression feature is propagated all the way down the video, helping enhance features of low-quality frames. This impression mechanism makes it possible to perform long-range multi-frame feature fusion among sparse keyframes with minimal overhead. It significantly improves per-frame detection baseline on ImageNet VID while being 3 times faster (20 fps). We hope Impression Network can provide a new perspective on video feature enhancement. Code will be made available.