EgoReID Dataset: Person Re-identification in Videos Acquired by Mobile Devices with First-Person Point-of-View

TL;DR

This paper introduces EgoReID, a new egocentric video dataset captured by mobile devices, and proposes a framework combining visual and sensor data to improve person re-identification in first-person videos.

Contribution

The paper provides the first egocentric person ReID dataset with sensor metadata and develops a novel method leveraging both visual features and sensor data for enhanced ReID performance.

Findings

Sensor data significantly reduces search space.

The method improves ReID accuracy in egocentric videos.

The dataset enables research on first-person person ReID challenges.

Abstract

In recent years, we have seen the performance of video-based person Re-Identification (ReID) methods have improved considerably. However, most of the work in this area has dealt with videos acquired by fixed cameras with wider field of view. Recently, widespread use of wearable cameras and recording devices such as cellphones have opened the door to interesting research in first-person Point-of-view (POV) videos (egocentric videos). Nonetheless, analysis of such videos is challenging due to factors such as poor video quality due to ego-motion, blurriness, severe changes in lighting conditions and perspective distortions. To facilitate the research towards conquering these challenges, this paper contributes a new dataset called EgoReID. The dataset is captured using 3 mobile cellphones with non-overlapping field-of-view. It contains 900 IDs and around 10,200 tracks with a total of 176,000 detections. The dataset also contains 12-sensor meta data e.g. camera orientation pitch and rotation for each video. In addition, we propose a new framework which takes advantage of both visual and sensor meta data to successfully perform Person ReID. We extend image-based re-ID method employing human body parsing trained on ten datasets to video-based re-ID. In our method, first frame level local features are extracted for each semantic region, then 3D convolutions are applied to encode the temporal information in each sequence of semantic regions. Additionally, we employ sensor meta data to predict targets' next camera and their estimated time of arrival, which considerably improves our ReID performance as it significantly reduces our search space.