NavAI: A Generalizable LLM Framework for Navigation Tasks in Virtual Reality Environments

TL;DR

NavAI introduces a versatile LLM-based framework for navigation in VR environments, achieving high success rates in goal-oriented tasks and addressing the limitations of existing path optimization methods.

Contribution

The paper presents NavAI, a novel LLM-based navigation framework that generalizes across diverse VR environments and supports complex goal-directed actions.

Findings

Achieves 89% success rate in goal-oriented VR navigation tasks

Demonstrates effectiveness across three distinct VR environments

Identifies current limitations of LLM reliance in dynamic scenarios

Abstract

Navigation is one of the fundamental tasks for automated exploration in Virtual Reality (VR). Existing technologies primarily focus on path optimization in 360-degree image datasets and 3D simulators, which cannot be directly applied to immersive VR environments. To address this gap, we present NavAI, a generalizable large language model (LLM)-based navigation framework that supports both basic actions and complex goal-directed tasks across diverse VR applications. We evaluate NavAI in three distinct VR environments through goal-oriented and exploratory tasks. Results show that it achieves high accuracy, with an 89% success rate in goal-oriented tasks. Our analysis also highlights current limitations of relying entirely on LLMs, particularly in scenarios that require dynamic goal assessment. Finally, we discuss the limitations observed during the experiments and offer insights for future research directions.