Adaptive Visual Navigation Assistant in 3D RPGs

TL;DR

This paper introduces a deep learning approach to detect and select critical map transition points in 3D RPGs, aiding navigation and level design, with a focus on robustness in low-data scenarios.

Contribution

It formalizes the detection of Spatial Transition Points and the selection of the Main STP, providing a new dataset, baseline pipeline, and insights into model adaptation strategies.

Findings

Adapter-only transfer is more robust in low-data scenarios.

Full-network fine-tuning yields better detection with sufficient data.

The approach establishes a baseline for future AI navigation tools.

Abstract

In complex 3D game environments, players rely on visual affordances to spot map transition points. Efficient identification of such points is important to client-side auto-mapping, and provides an objective basis for evaluating map cue presentation. In this work, we formalize the task of detecting traversable Spatial Transition Points (STPs)-connectors between two sub regions-and selecting the singular Main STP (MSTP), the unique STP that lies on the designer-intended critical path toward the player's current macro-objective, from a single game frame, proposing this as a new research focus. We introduce a two-stage deep-learning pipeline that first detects potential STPs using Faster R-CNN and then ranks them with a lightweight MSTP selector that fuses local and global visual features. Both stages benefit from parameter-efficient adapters, and we further introduce an optional retrieval-augmented fusion step. Our primary goal is to establish the feasibility of this problem and set baseline performance metrics. We validate our approach on a custom-built, diverse dataset collected from five Action RPG titles. Our experiments reveal a key trade-off: while full-network fine-tuning produces superior STP detection with sufficient data, adapter-only transfer is significantly more robust and effective in low-data scenarios and for the MSTP selection task. By defining this novel problem, providing a baseline pipeline and dataset, and offering initial insights into efficient model adaptation, we aim to contribute to future AI-driven navigation aids and data-informed level-design tools.