OLiVia-Nav: An Online Lifelong Vision Language Approach for Mobile Robot Social Navigation

TL;DR

OLiVia-Nav is a novel online lifelong vision language framework that enables social navigation for robots by integrating large vision-language models with continual learning, improving adaptability and social compliance in dynamic environments.

Contribution

The paper introduces SC-CLIP, a distillation method that transfers social reasoning from large VLMs to lightweight models, enabling real-time social navigation adaptation.

Findings

OLiVia-Nav outperforms state-of-the-art methods in social navigation metrics.

The approach effectively encodes social and environmental context during navigation.

Ablation studies confirm the importance of each component in the system.

Abstract

Service robots in human-centered environments such as hospitals, office buildings, and long-term care homes need to navigate while adhering to social norms to ensure the safety and comfortability of the people they are sharing the space with. Furthermore, they need to adapt to new social scenarios that can arise during robot navigation. In this paper, we present a novel Online Lifelong Vision Language architecture, OLiVia- Nav, which uniquely integrates vision-language models (VLMs) with an online lifelong learning framework for robot social navigation. We introduce a unique distillation approach, Social Context Contrastive Language Image Pre-training (SC-CLIP), to transfer the social reasoning capabilities of large VLMs to a lightweight VLM, in order for OLiVia-Nav to directly encode social and environment context during robot navigation. These encoded embeddings are used to generate and select robot social compliant trajectories. The lifelong learning capabilities of SC-CLIP enable OLiVia-Nav to update the robot trajectory planning overtime as new social scenarios are encountered. We conducted extensive real-world experiments in diverse social navigation scenarios. The results showed that OLiVia-Nav outperformed existing state-of-the-art DRL and VLM methods in terms of mean squared error, Hausdorff loss, and personal space violation duration. Ablation studies also verified the design choices for OLiVia-Nav.