A Generic Object Re-identification System for Short Videos

TL;DR

This paper introduces a comprehensive object re-identification system tailored for short videos, combining novel detection, tracking, and re-identification modules to improve accuracy and efficiency in complex visual environments.

Contribution

The paper presents a unified system with a Temporal Information Fusion Network and Cross-Layer Pointwise Siamese Network, advancing object detection and tracking in short videos.

Findings

Comparable accuracy to state-of-the-art detectors

Improved time efficiency in object detection

Enhanced robustness of tracking in short videos

Abstract

Short video applications like TikTok and Kwai have been a great hit recently. In order to meet the increasing demands and take full advantage of visual information in short videos, objects in each short video need to be located and analyzed as an upstream task. A question is thus raised -- how to improve the accuracy and robustness of object detection, tracking, and re-identification across tons of short videos with hundreds of categories and complicated visual effects (VFX). To this end, a system composed of a detection module, a tracking module and a generic object re-identification module, is proposed in this paper, which captures features of major objects from short videos. In particular, towards the high efficiency demands in practical short video application, a Temporal Information Fusion Network (TIFN) is proposed in the object detection module, which shows comparable accuracy and improved time efficiency to the state-of-the-art video object detector. Furthermore, in order to mitigate the fragmented issue of tracklets in short videos, a Cross-Layer Pointwise Siamese Network (CPSN) is proposed in the tracking module to enhance the robustness of the appearance model. Moreover, in order to evaluate the proposed system, two challenge datasets containing real-world short videos are built for video object trajectory extraction and generic object re-identification respectively. Overall, extensive experiments for each module and the whole system demonstrate the effectiveness and efficiency of our system.