CorVS: Person Identification via Video Trajectory-Sensor Correspondence in a Real-World Warehouse

TL;DR

CorVS is a novel deep learning approach that matches visual trajectories with sensor data to identify workers in real-world warehouse environments, overcoming limitations of appearance-based methods.

Contribution

This paper introduces CorVS, a data-driven person identification method that combines visual and sensor data using deep learning, specifically designed for real-world warehouse conditions.

Findings

Effective in real-world warehouse scenarios

Improves accuracy over appearance-based methods

Demonstrates robustness with actual operational data

Abstract

Worker location data is key to higher productivity in industrial sites. Cameras are a promising tool for localization in logistics warehouses since they also offer valuable environmental contexts such as package status. However, identifying individuals with only visual data is often impractical. Accordingly, several prior studies identified people in videos by comparing their trajectories and wearable sensor measurements. While this approach has advantages such as independence from appearance, the existing methods may break down under real-world conditions. To overcome this challenge, we propose CorVS, a novel data-driven person identification method based on correspondence between visual tracking trajectories and sensor measurements. Firstly, our deep learning model predicts correspondence probabilities and reliabilities for every pair of a trajectory and sensor measurements. Secondly, our algorithm matches the trajectories and sensor measurements over time using the predicted probabilities and reliabilities. We developed a dataset with actual warehouse operations and demonstrated the method's effectiveness for real-world applications.