Enhancing Situational Awareness in Underwater Robotics with Multi-modal Spatial Perception

TL;DR

This paper introduces a multi-modal sensing approach combining cameras, IMUs, and acoustic sensors to improve underwater SLAM, achieving robust real-time 3D mapping in challenging underwater environments.

Contribution

It presents a novel multi-sensor fusion framework for underwater SLAM that integrates geometric, learning-based, and semantic techniques, validated through real-world field deployments.

Findings

Real-time state estimation achieved in challenging conditions

High-quality 3D reconstructions demonstrated

Multi-modal fusion improves robustness over single-sensor methods

Abstract

Autonomous Underwater Vehicles (AUVs) and Remotely Operated Vehicles (ROVs) demand robust spatial perception capabilities, including Simultaneous Localization and Mapping (SLAM), to support both remote and autonomous tasks. Vision-based systems have been integral to these advancements, capturing rich color and texture at low cost while enabling semantic scene understanding. However, underwater conditions -- such as light attenuation, backscatter, and low contrast -- often degrade image quality to the point where traditional vision-based SLAM pipelines fail. Moreover, these pipelines typically rely on monocular or stereo inputs, limiting their scalability to the multi-camera configurations common on many vehicles. To address these issues, we propose to leverage multi-modal sensing that fuses data from multiple sensors-including cameras, inertial measurement units (IMUs), and acoustic devices-to enhance situational awareness and enable robust, real-time SLAM. We explore both geometric and learning-based techniques along with semantic analysis, and conduct experiments on the data collected from a work-class ROV during several field deployments in the Trondheim Fjord. Through our experimental results, we demonstrate the feasibility of real-time reliable state estimation and high-quality 3D reconstructions in visually challenging underwater conditions. We also discuss system constraints and identify open research questions, such as sensor calibration, limitations with learning-based methods, that merit further exploration to advance large-scale underwater operations.