ChatSim: Underwater Simulation with Natural Language Prompting

TL;DR

ChatSim integrates large language models with underwater simulation to enable intuitive environment control via natural language, facilitating cost-effective testing for marine exploration and research.

Contribution

It introduces a novel method combining LLMs with OysterSim to allow natural language control of underwater simulations, enhancing usability and flexibility.

Findings

Enables intuitive environment creation through natural language prompts.

Reduces time and cost for underwater simulation testing.

Improves simulation realism and control for marine research.

Abstract

Robots are becoming an essential part of many operations including marine exploration or environmental monitoring. However, the underwater environment presents many challenges, including high pressure, limited visibility, and harsh conditions that can damage equipment. Real-world experimentation can be expensive and difficult to execute. Therefore, it is essential to simulate the performance of underwater robots in comparable environments to ensure their optimal functionality within practical real-world contexts.OysterSim generates photo-realistic images and segmentation masks of objects in marine environments, providing valuable training data for underwater computer vision applications. By integrating ChatGPT into underwater simulations, users can convey their thoughts effortlessly and intuitively create desired underwater environments without intricate coding. \invis{Moreover, researchers can realize substantial time and cost savings by evaluating their algorithms across diverse underwater conditions in the simulation.} The objective of ChatSim is to integrate Large Language Models (LLM) with a simulation environment~(OysterSim), enabling direct control of the simulated environment via natural language input. This advancement can greatly enhance the capabilities of underwater simulation, with far-reaching benefits for marine exploration and broader scientific research endeavors.