Semantic-Based VPS for Smartphone Localization in Challenging Urban Environments

TL;DR

This paper introduces a semantic-based visual positioning system (VPS) utilizing 3D city models and material segmentation to improve smartphone localization accuracy in challenging urban environments where GNSS signals are unreliable.

Contribution

It proposes a novel semantic-based VPS that leverages city models and material segmentation for more accurate pose estimation in urban canyons, outperforming existing methods.

Findings

Achieves 2.0m accuracy in high-rise urban streets

Attains 5.5m accuracy in foliage-dense environments

Improves yaw estimation to 2.3 degrees, eight times better than IMU

Abstract

Accurate smartphone-based outdoor localization system in deep urban canyons are increasingly needed for various IoT applications such as augmented reality, intelligent transportation, etc. The recently developed feature-based visual positioning system (VPS) by Google detects edges from smartphone images to match with pre-surveyed edges in their map database. As smart cities develop, the building information modeling (BIM) becomes widely available, which provides an opportunity for a new semantic-based VPS. This article proposes a novel 3D city model and semantic-based VPS for accurate and robust pose estimation in urban canyons where global navigation satellite system (GNSS) tends to fail. In the offline stage, a material segmented city model is used to generate segmented images. In the online stage, an image is taken with a smartphone camera that provides textual information about the surrounding environment. The approach utilizes computer vision algorithms to rectify and hand segment between the different types of material identified in the smartphone image. A semantic-based VPS method is then proposed to match the segmented generated images with the segmented smartphone image. Each generated image holds a pose that contains the latitude, longitude, altitude, yaw, pitch, and roll. The candidate with the maximum likelihood is regarded as the precise pose of the user. The positioning results achieves 2.0m level accuracy in common high rise along street, 5.5m in foliage dense environment and 15.7m in alleyway. A 45% positioning improvement to current state-of-the-art method. The estimation of yaw achieves 2.3{\deg} level accuracy, 8 times the improvement to smartphone IMU.