Vision-Based Localization and LLM-based Navigation for Indoor Environments

TL;DR

This paper introduces a novel indoor navigation system combining vision-based localization with LLM-guided navigation, achieving high accuracy in complex environments without relying on GPS or extensive infrastructure.

Contribution

It presents a new integrated approach using CNN-based localization and LLM-driven navigation, demonstrating effective indoor positioning and routing with off-the-shelf devices.

Findings

Localization accuracy of 96% in challenging environments

Average navigation instruction accuracy of 75% with ChatGPT

Scalable, infrastructure-free indoor navigation solution

Abstract

Indoor navigation remains a complex challenge due to the absence of reliable GPS signals and the architectural intricacies of large enclosed environments. This study presents an indoor localization and navigation approach that integrates vision-based localization with large language model (LLM)-based navigation. The localization system utilizes a ResNet-50 convolutional neural network fine-tuned through a two-stage process to identify the user's position using smartphone camera input. To complement localization, the navigation module employs an LLM, guided by a carefully crafted system prompt, to interpret preprocessed floor plan images and generate step-by-step directions. Experimental evaluation was conducted in a realistic office corridor with repetitive features and limited visibility to test localization robustness. The model achieved high confidence and an accuracy of 96% across all tested waypoints, even under constrained viewing conditions and short-duration queries. Navigation tests using ChatGPT on real building floor maps yielded an average instruction accuracy of 75%, with observed limitations in zero-shot reasoning and inference time. This research demonstrates the potential for scalable, infrastructure-free indoor navigation using off-the-shelf cameras and publicly available floor plans, particularly in resource-constrained settings like hospitals, airports, and educational institutions.