Statistics of RGBD Images

TL;DR

This paper investigates the statistical properties of RGBD images using probabilistic models, demonstrating that learned generative models can effectively enhance depth quality from color information, even with synthetic training data.

Contribution

It introduces a statistical approach to depth enhancement using generative models trained on synthetic images, outperforming existing methods.

Findings

Learned generative models outperform state-of-the-art methods.

Synthetic training data can effectively improve depth quality.

Probabilistic models capture natural image statistics for depth enhancement.

Abstract

Cameras that can measure the depth of each pixel in addition to its color have become easily available and are used in many consumer products worldwide. Often the depth channel is captured at lower quality compared to the RGB channels and different algorithms have been proposed to improve the quality of the D channel given the RGB channels. Typically these approaches work by assuming that edges in RGB are correlated with edges in D. In this paper we approach this problem from the standpoint of natural image statistics. We obtain examples of high quality RGBD images from a computer graphics generated movie (MPI-Sintel) and we use these examples to compare different probabilistic generative models of RGBD image patches. We then use the generative models together with a degradation model and obtain a Bayes Least Squares (BLS) estimator of the D channel given the RGB channels. Our results show that learned generative models outperform the state-of-the-art in improving the quality of depth channels given the color channels in natural images even when training is performed on artificially generated images.