Cross-Modal Synergies: Unveiling the Potential of Motion-Aware Fusion Networks in Handling Dynamic and Static ReID Scenarios

TL;DR

This paper presents MOTAR-FUSE, a motion-aware fusion network that leverages motion cues from images and videos to improve person re-identification, especially under occlusions, demonstrating superior benchmark performance.

Contribution

Introduces a novel Motion-Aware Fusion network with a dual-input visual adapter and motion consistency task for enhanced ReID in complex scenarios.

Findings

Outperforms existing methods on multiple ReID benchmarks.

Effectively captures human motion dynamics under occlusions.

Enhances feature extraction through motion-aware transformer.

Abstract

Navigating the complexities of person re-identification (ReID) in varied surveillance scenarios, particularly when occlusions occur, poses significant challenges. We introduce an innovative Motion-Aware Fusion (MOTAR-FUSE) network that utilizes motion cues derived from static imagery to significantly enhance ReID capabilities. This network incorporates a dual-input visual adapter capable of processing both images and videos, thereby facilitating more effective feature extraction. A unique aspect of our approach is the integration of a motion consistency task, which empowers the motion-aware transformer to adeptly capture the dynamics of human motion. This technique substantially improves the recognition of features in scenarios where occlusions are prevalent, thereby advancing the ReID process. Our comprehensive evaluations across multiple ReID benchmarks, including holistic, occluded, and video-based scenarios, demonstrate that our MOTAR-FUSE network achieves superior performance compared to existing approaches.