Learning Pseudo Front Depth for 2D Forward-Looking Sonar-based Multi-view Stereo

TL;DR

This paper introduces a novel learning-based multi-view stereo approach for 2D forward-looking sonar images, using a pseudo front depth representation to accurately estimate 3D underwater scenes with minimal viewpoints.

Contribution

The work proposes a new elevation plane sweeping method and pseudo front depth concept to improve 3D reconstruction from limited sonar views, addressing ambiguity issues in acoustic imaging.

Findings

High-accuracy 3D reconstructions with only two or three images.

Synthetic datasets and a real large-scale water tank dataset were used for validation.

The method outperforms existing state-of-the-art techniques.

Abstract

Retrieving the missing dimension information in acoustic images from 2D forward-looking sonar is a well-known problem in the field of underwater robotics. There are works attempting to retrieve 3D information from a single image which allows the robot to generate 3D maps with fly-through motion. However, owing to the unique image formulation principle, estimating 3D information from a single image faces severe ambiguity problems. Classical methods of multi-view stereo can avoid the ambiguity problems, but may require a large number of viewpoints to generate an accurate model. In this work, we propose a novel learning-based multi-view stereo method to estimate 3D information. To better utilize the information from multiple frames, an elevation plane sweeping method is proposed to generate the depth-azimuth-elevation cost volume. The volume after regularization can be considered as a probabilistic volumetric representation of the target. Instead of performing regression on the elevation angles, we use pseudo front depth from the cost volume to represent the 3D information which can avoid the 2D-3D problem in acoustic imaging. High-accuracy results can be generated with only two or three images. Synthetic datasets were generated to simulate various underwater targets. We also built the first real dataset with accurate ground truth in a large scale water tank. Experimental results demonstrate the superiority of our method, compared to other state-of-the-art methods.