What-and-Where to Match: Deep Spatially Multiplicative Integration Networks for Person Re-identification

TL;DR

This paper introduces a novel deep neural network for person re-identification that dynamically emphasizes matching relevant local patterns and aligns spatial features to improve accuracy across camera views.

Contribution

It proposes a deep multiplicative integration gating function combined with spatially recurrent pooling, enabling flexible and spatially aligned local feature matching for person re-id.

Findings

Outperforms existing methods on VIPeR, CUHK03, and Market-1501 datasets.

Effectively captures local patterns and spatial dependencies.

End-to-end trainable network enhances person re-identification accuracy.

Abstract

Matching pedestrians across disjoint camera views, known as person re-identification (re-id), is a challenging problem that is of importance to visual recognition and surveillance. Most existing methods exploit local regions within spatial manipulation to perform matching in local correspondence. However, they essentially extract \emph{fixed} representations from pre-divided regions for each image and perform matching based on the extracted representation subsequently. For models in this pipeline, local finer patterns that are crucial to distinguish positive pairs from negative ones cannot be captured, and thus making them underperformed. In this paper, we propose a novel deep multiplicative integration gating function, which answers the question of \emph{what-and-where to match} for effective person re-id. To address \emph{what} to match, our deep network emphasizes common local patterns by learning joint representations in a multiplicative way. The network comprises two Convolutional Neural Networks (CNNs) to extract convolutional activations, and generates relevant descriptors for pedestrian matching. This thus, leads to flexible representations for pair-wise images. To address \emph{where} to match, we combat the spatial misalignment by performing spatially recurrent pooling via a four-directional recurrent neural network to impose spatial dependency over all positions with respect to the entire image. The proposed network is designed to be end-to-end trainable to characterize local pairwise feature interactions in a spatially aligned manner. To demonstrate the superiority of our method, extensive experiments are conducted over three benchmark data sets: VIPeR, CUHK03 and Market-1501.