Color-Guided Flying Pixel Correction in Depth Images

TL;DR

This paper introduces a new method for correcting flying pixel artifacts in ToF depth images by utilizing aligned RGB data to accurately estimate true depth values through an iterative optimization process.

Contribution

It presents a novel RGB-guided correction algorithm for flying pixels in ToF sensors, improving depth accuracy by leveraging RGB-D data and a two-step iterative approach.

Findings

Accurately estimates true depth of flying pixels.

Performs comparably to existing algorithms in accuracy.

Uses a two-step iterative correction process.

Abstract

We present a novel method to correct flying pixels within data captured by Time-of-flight (ToF) sensors. Flying pixel (FP) artifacts occur when signals from foreground and background objects reach the same sensor pixel, leading to a confident yet incorrect depth estimation in space - floating between two objects. Commercial RGB-D cameras have a complementary setup consisting of ToF sensors to capture depth in addition to RGB cameras. We propose a novel method to correct FPs by leveraging the aligned RGB and depth image in such RGB-D cameras to estimate the true depth values of FPs. Our method defines a 3D neighborhood around each point, representing a "field of view" that mirrors the acquisition process of ToF cameras. We propose a two-step iterative correction algorithm in which the FPs are first identified. Then, we estimate the true depth value of FPs by solving a least-squares optimization problem. Experimental results show that our proposed algorithm estimates the depth value of FPs as accurately as other algorithms in the literature.