Where-and-When to Look: Deep Siamese Attention Networks for Video-based Person Re-identification

TL;DR

This paper introduces a deep Siamese attention network that jointly learns spatiotemporal features and similarity metrics for video-based person re-identification, effectively focusing on relevant regions across frames to improve matching accuracy.

Contribution

It proposes a novel Siamese attention architecture that integrates spatial and temporal attention mechanisms within a unified model for person re-id.

Findings

Outperforms state-of-the-art methods on benchmark datasets

Effectively captures discriminative local features in videos

Jointly learns feature representations and similarity metrics

Abstract

Video-based person re-identification (re-id) is a central application in surveillance systems with significant concern in security. Matching persons across disjoint camera views in their video fragments is inherently challenging due to the large visual variations and uncontrolled frame rates. There are two steps crucial to person re-id, namely discriminative feature learning and metric learning. However, existing approaches consider the two steps independently, and they do not make full use of the temporal and spatial information in videos. In this paper, we propose a Siamese attention architecture that jointly learns spatiotemporal video representations and their similarity metrics. The network extracts local convolutional features from regions of each frame, and enhance their discriminative capability by focusing on distinct regions when measuring the similarity with another pedestrian video. The attention mechanism is embedded into spatial gated recurrent units to selectively propagate relevant features and memorize their spatial dependencies through the network. The model essentially learns which parts (\emph{where}) from which frames (\emph{when}) are relevant and distinctive for matching persons and attaches higher importance therein. The proposed Siamese model is end-to-end trainable to jointly learn comparable hidden representations for paired pedestrian videos and their similarity value. Extensive experiments on three benchmark datasets show the effectiveness of each component of the proposed deep network while outperforming state-of-the-art methods.