Mobile Video Object Detection with Temporally-Aware Feature Maps

TL;DR

This paper presents a real-time, mobile-friendly video object detection model that integrates convolutional LSTMs with a novel Bottleneck-LSTM layer to enhance temporal feature propagation efficiently.

Contribution

It introduces an efficient Bottleneck-LSTM layer and a recurrent-convolutional architecture for fast, accurate video object detection on low-powered devices.

Findings

Achieves up to 15 FPS on mobile CPU

Outperforms existing fast detection models in size and speed

Maintains accuracy comparable to high-cost models

Abstract

This paper introduces an online model for object detection in videos designed to run in real-time on low-powered mobile and embedded devices. Our approach combines fast single-image object detection with convolutional long short term memory (LSTM) layers to create an interweaved recurrent-convolutional architecture. Additionally, we propose an efficient Bottleneck-LSTM layer that significantly reduces computational cost compared to regular LSTMs. Our network achieves temporal awareness by using Bottleneck-LSTMs to refine and propagate feature maps across frames. This approach is substantially faster than existing detection methods in video, outperforming the fastest single-frame models in model size and computational cost while attaining accuracy comparable to much more expensive single-frame models on the Imagenet VID 2015 dataset. Our model reaches a real-time inference speed of up to 15 FPS on a mobile CPU.