ViFi-Loc: Multi-modal Pedestrian Localization using GAN with Camera-Phone Correspondences

TL;DR

This paper introduces ViFi-Loc, a GAN-based model that improves pedestrian localization accuracy in smart city environments by learning camera-phone data linkages and supporting self-learning, achieving 1-2 meter error across diverse outdoor scenes.

Contribution

The paper presents a novel GAN architecture that learns camera-phone data correspondences and enhances pedestrian localization without requiring data association during inference.

Findings

Achieves 1-2 meter localization error in outdoor scenes.

Supports self-learning and automatic data collection.

Improves accuracy by up to 26% after fine-tuning.

Abstract

In Smart City and Vehicle-to-Everything (V2X) systems, acquiring pedestrians' accurate locations is crucial to traffic safety. Current systems adopt cameras and wireless sensors to detect and estimate people's locations via sensor fusion. Standard fusion algorithms, however, become inapplicable when multi-modal data is not associated. For example, pedestrians are out of the camera field of view, or data from camera modality is missing. To address this challenge and produce more accurate location estimations for pedestrians, we propose a Generative Adversarial Network (GAN) architecture. During training, it learns the underlying linkage between pedestrians' camera-phone data correspondences. During inference, it generates refined position estimations based only on pedestrians' phone data that consists of GPS, IMU and FTM. Results show that our GAN produces 3D coordinates at 1 to 2 meter localization error across 5 different outdoor scenes. We further show that the proposed model supports self-learning. The generated coordinates can be associated with pedestrian's bounding box coordinates to obtain additional camera-phone data correspondences. This allows automatic data collection during inference. After fine-tuning on the expanded dataset, localization accuracy is improved by up to 26%.