Video-based Person Re-identification with Long Short-Term Representation Learning

TL;DR

This paper introduces a novel deep learning framework, LSTRL, that combines long-term and short-term video features for improved person re-identification across non-overlapping camera views, demonstrating superior performance on benchmark datasets.

Contribution

The paper proposes a new framework with MAE and BME modules for effective long- and short-term feature extraction in video-based person re-identification, which can be integrated into existing models.

Findings

Outperforms most state-of-the-art methods on benchmark datasets.

Effectively captures multi-granularity appearance features.

Efficiently extracts reciprocal motion information from frames.

Abstract

Video-based person Re-Identification (V-ReID) aims to retrieve specific persons from raw videos captured by non-overlapped cameras. As a fundamental task, it spreads many multimedia and computer vision applications. However, due to the variations of persons and scenes, there are still many obstacles that must be overcome for high performance. In this work, we notice that both the long-term and short-term information of persons are important for robust video representations. Thus, we propose a novel deep learning framework named Long Short-Term Representation Learning (LSTRL) for effective V-ReID. More specifically, to extract long-term representations, we propose a Multi-granularity Appearance Extractor (MAE), in which four granularity appearances are effectively captured across multiple frames. Meanwhile, to extract short-term representations, we propose a Bi-direction Motion Estimator (BME), in which reciprocal motion information is efficiently extracted from consecutive frames. The MAE and BME are plug-and-play and can be easily inserted into existing networks for efficient feature learning. As a result, they significantly improve the feature representation ability for V-ReID. Extensive experiments on three widely used benchmarks show that our proposed approach can deliver better performances than most state-of-the-arts.