RADU: Ray-Aligned Depth Update Convolutions for ToF Data Denoising

TL;DR

This paper introduces RADU, a 3D-aware neural network for denoising ToF camera data, leveraging ray-aligned convolutions and self-training to improve performance on real and synthetic datasets.

Contribution

The paper presents a novel 3D point convolution method for ToF data denoising, incorporating ray-aligned convolutions and self-training on unlabeled data.

Findings

Outperforms SOTA on multiple datasets

Effective in real-world and synthetic scenarios

Utilizes self-training to adapt to real-world data

Abstract

Time-of-Flight (ToF) cameras are subject to high levels of noise and distortions due to Multi-Path-Interference (MPI). While recent research showed that 2D neural networks are able to outperform previous traditional State-of-the-Art (SOTA) methods on denoising ToF-Data, little research on learning-based approaches has been done to make direct use of the 3D information present in depth images. In this paper, we propose an iterative denoising approach operating in 3D space, that is designed to learn on 2.5D data by enabling 3D point convolutions to correct the points' positions along the view direction. As labeled real world data is scarce for this task, we further train our network with a self-training approach on unlabeled real world data to account for real world statistics. We demonstrate that our method is able to outperform SOTA methods on several datasets, including two real world datasets and a new large-scale synthetic data set introduced in this paper.