DEFLOW: Self-supervised 3D Motion Estimation of Debris Flow

TL;DR

DEFLOW is a self-supervised deep learning model designed for 3D motion estimation of debris flows, utilizing a new dataset and multi-level sensor fusion to achieve state-of-the-art results in natural scene flow estimation.

Contribution

The paper introduces DEFLOW, a novel self-supervised model with multi-level sensor fusion and multi-frame processing for debris flow motion estimation, along with a new dataset.

Findings

Achieves state-of-the-art optical flow and depth estimation on debris flow data.

Fully automates debris flow motion estimation.

Introduces a new dataset for natural scene flow in debris flows.

Abstract

Existing work on scene flow estimation focuses on autonomous driving and mobile robotics, while automated solutions are lacking for motion in nature, such as that exhibited by debris flows. We propose DEFLOW, a model for 3D motion estimation of debris flows, together with a newly captured dataset. We adopt a novel multi-level sensor fusion architecture and self-supervision to incorporate the inductive biases of the scene. We further adopt a multi-frame temporal processing module to enable flow speed estimation over time. Our model achieves state-of-the-art optical flow and depth estimation on our dataset, and fully automates the motion estimation for debris flows. The source code and dataset are available at project page.