Towards Differentiable Rendering for Sidescan Sonar Imagery

TL;DR

This paper introduces a differentiable renderer tailored for sidescan sonar imagery, enabling 3D seafloor reconstruction from 2D data using gradient-based optimization and deep learning techniques.

Contribution

It presents the first differentiable renderer for sidescan sonar images, facilitating 3D reconstruction directly from 2D sonar data with reduced need for 3D annotations.

Findings

Successful 3D seafloor reconstruction from 2D sonar data

Integration of deep learning with differentiable rendering for sonar imagery

Reduction in data annotation requirements

Abstract

Recent advances in differentiable rendering, which allow calculating the gradients of 2D pixel values with respect to 3D object models, can be applied to estimation of the model parameters by gradient-based optimization with only 2D supervision. It is easy to incorporate deep neural networks into such an optimization pipeline, allowing the leveraging of deep learning techniques. This also largely reduces the requirement for collecting and annotating 3D data, which is very difficult for applications, for example when constructing geometry from 2D sensors. In this work, we propose a differentiable renderer for sidescan sonar imagery. We further demonstrate its ability to solve the inverse problem of directly reconstructing a 3D seafloor mesh from only 2D sidescan sonar data.