Neural reconstruction of 3D ocean wave hydrodynamics from camera sensing

TL;DR

This paper introduces a neural network for efficient 3D ocean wave reconstruction from camera data, achieving high accuracy and real-time performance while handling occlusions and complex wave dynamics.

Contribution

The proposed attention-augmented pyramid neural network enables fast, accurate, and occlusion-robust 3D wave surface and velocity field reconstruction using physics-based constraints.

Findings

Millimeter-level wave elevation prediction

Reconstruction of two million points in 1.35 seconds

Outperforms conventional visual reconstruction methods

Abstract

Precise three-dimensional (3D) reconstruction of wave free surfaces and associated velocity fields is essential for developing a comprehensive understanding of ocean physics. To address the high computational cost of dense visual reconstruction in long-term ocean wave observation tasks and the challenges introduced by persistent visual occlusions, we propose an wave free surface visual reconstruction neural network, which is designed as an attention-augmented pyramid architecture tailored to the multi-scale and temporally continuous characteristics of wave motions. Using physics-based constraints, we perform time-resolved reconstruction of nonlinear 3D velocity fields from the evolving free-surface boundary. Experiments under real-sea conditions demonstrate millimetre-level wave elevation prediction in the central region, dominant-frequency errors below 0.01 Hz, precise estimation of high-frequency spectral power laws, and high-fidelity 3D reconstruction of nonlinear velocity fields, while enabling dense reconstruction of two million points in only 1.35 s. Built on a stereo-vision dataset, the model outperforms conventional visual reconstruction approaches and maintains strong generalization in occluded conditions, owing to its global multi-scale attention and its learned encoding of wave propagation dynamics.