Learning the Correction for Multi-Path Deviations in Time-of-Flight Cameras

TL;DR

This paper presents a machine learning approach using Random Forests to correct multipath deviations in ToF camera data, significantly improving depth accuracy by reducing errors from 19% to 3%.

Contribution

It introduces new datasets and demonstrates that machine learning can effectively correct multipath effects in ToF cameras.

Findings

Reduced per-pixel error from 19% to 3%.

Lowered variance of depth errors by an order of magnitude.

Validated effectiveness of Random Forests for ToF correction.

Abstract

The Multipath effect in Time-of-Flight(ToF) cameras still remains to be a challenging problem that hinders further processing of 3D data information. Based on the evidence from previous literature, we explored the possibility of using machine learning techniques to correct this effect. Firstly, we created two new datasets of of ToF images rendered via ToF simulator of LuxRender. These two datasets contain corners in multiple orientations and with different material properties. We chose scenes with corners as multipath effects are most pronounced in corners. Secondly, we used this dataset to construct a learning model to predict real valued corrections to the ToF data using Random Forests. We found out that in our smaller dataset we were able to predict real valued correction and improve the quality of depth images significantly by removing multipath bias. With our algorithm, we improved relative per-pixel error from average value of 19% to 3%. Additionally, variance of the error was lowered by an order of magnitude.