T-CNN: Tubelets with Convolutional Neural Networks for Object Detection from Videos

TL;DR

T-CNN introduces a deep learning framework that leverages temporal and contextual information from tubelets to significantly enhance object detection performance in videos, outperforming existing methods in the ILSVRC2015 challenge.

Contribution

The paper presents T-CNN, a novel framework that effectively incorporates temporal and contextual cues from video tubelets into object detection, improving accuracy over traditional still-image detectors.

Findings

T-CNN achieved top performance in the ILSVRC2015 VID task.

Incorporating tubelet information boosts detection accuracy in videos.

The framework outperforms previous state-of-the-art methods.

Abstract

The state-of-the-art performance for object detection has been significantly improved over the past two years. Besides the introduction of powerful deep neural networks such as GoogleNet and VGG, novel object detection frameworks such as R-CNN and its successors, Fast R-CNN and Faster R-CNN, play an essential role in improving the state-of-the-art. Despite their effectiveness on still images, those frameworks are not specifically designed for object detection from videos. Temporal and contextual information of videos are not fully investigated and utilized. In this work, we propose a deep learning framework that incorporates temporal and contextual information from tubelets obtained in videos, which dramatically improves the baseline performance of existing still-image detection frameworks when they are applied to videos. It is called T-CNN, i.e. tubelets with convolutional neueral networks. The proposed framework won the recently introduced object-detection-from-video (VID) task with provided data in the ImageNet Large-Scale Visual Recognition Challenge 2015 (ILSVRC2015).