PIG-Nav: Key Insights for Pretrained Image Goal Navigation Models

TL;DR

PIG-Nav introduces improved pretraining strategies for vision-based robotic navigation, utilizing early-fusion networks and auxiliary tasks, leading to significant performance gains in diverse environments with minimal fine-tuning.

Contribution

The paper presents novel pretraining techniques and a data preprocessing pipeline that enhance zero-shot and fine-tuned navigation performance of vision-based models.

Findings

22.6% average improvement in zero-shot performance

37.5% improvement with fine-tuning over existing models

Effective in simulated and real-world environments

Abstract

Recent studies have explored pretrained (foundation) models for vision-based robotic navigation, aiming to achieve generalizable navigation and positive transfer across diverse environments while enhancing zero-shot performance in unseen settings. In this work, we introduce PIG-Nav (Pretrained Image-Goal Navigation), a new approach that further investigates pretraining strategies for vision-based navigation models and contributes in two key areas. Model-wise, we identify two critical design choices that consistently improve the performance of pretrained navigation models: (1) integrating an early-fusion network structure to combine visual observations and goal images via appropriately pretrained Vision Transformer (ViT) image encoder, and (2) introducing suitable auxiliary tasks to enhance global navigation representation learning, thus further improving navigation performance. Dataset-wise, we propose a novel data preprocessing pipeline for efficiently labeling large-scale game video datasets for navigation model training. We demonstrate that augmenting existing open navigation datasets with diverse gameplay videos improves model performance. Our model achieves an average improvement of 22.6% in zero-shot settings and a 37.5% improvement in fine-tuning settings over existing visual navigation foundation models in two complex simulated environments and one real-world environment. These results advance the state-of-the-art in pretrained image-goal navigation models. Notably, our model maintains competitive performance while requiring significantly less fine-tuning data, highlighting its potential for real-world deployment with minimal labeled supervision.