OceanChat: Piloting Autonomous Underwater Vehicles in Natural Language

TL;DR

OceanChat integrates large language models with task and motion planning to enable autonomous underwater vehicles to understand natural language commands and adapt to dynamic environments, improving operational success and efficiency.

Contribution

This work introduces OceanChat, a novel LLM-guided framework for AUVs that combines high-level goal translation, task grounding, real-time motion planning, and event-triggered replanning in underwater settings.

Findings

Enhanced success rate in AUV missions

Reduced computation time for planning

Robustness to environmental uncertainties

Abstract

In the trending research of fusing Large Language Models (LLMs) and robotics, we aim to pave the way for innovative development of AI systems that can enable Autonomous Underwater Vehicles (AUVs) to seamlessly interact with humans in an intuitive manner. We propose OceanChat, a system that leverages a closed-loop LLM-guided task and motion planning framework to tackle AUV missions in the wild. LLMs translate an abstract human command into a high-level goal, while a task planner further grounds the goal into a task sequence with logical constraints. To assist the AUV with understanding the task sequence, we utilize a motion planner to incorporate real-time Lagrangian data streams received by the AUV, thus mapping the task sequence into an executable motion plan. Considering the highly dynamic and partially known nature of the underwater environment, an event-triggered replanning scheme is developed to enhance the system's robustness towards uncertainty. We also build a simulation platform HoloEco that generates photo-realistic simulation for a wide range of AUV applications. Experimental evaluation verifies that the proposed system can achieve improved performance in terms of both success rate and computation time. Project website: \url{https://sites.google.com/view/oceanchat}