Depth Image Inpainting: Improving Low Rank Matrix Completion with Low Gradient Regularization

TL;DR

This paper introduces a novel depth image inpainting method that combines low rank and low gradient regularization, effectively handling depth images without color or temporal context by exploiting their unique gradient properties.

Contribution

It proposes a low gradient regularization technique tailored for depth images, enhancing low rank matrix completion for inpainting tasks.

Findings

Outperforms sparse gradient regularization in depth inpainting

Effectively captures gradual depth changes in images

Demonstrates improved inpainting quality on depth datasets

Abstract

We consider the case of inpainting single depth images. Without corresponding color images, previous or next frames, depth image inpainting is quite challenging. One natural solution is to regard the image as a matrix and adopt the low rank regularization just as inpainting color images. However, the low rank assumption does not make full use of the properties of depth images. A shallow observation may inspire us to penalize the non-zero gradients by sparse gradient regularization. However, statistics show that though most pixels have zero gradients, there is still a non-ignorable part of pixels whose gradients are equal to 1. Based on this specific property of depth images , we propose a low gradient regularization method in which we reduce the penalty for gradient 1 while penalizing the non-zero gradients to allow for gradual depth changes. The proposed low gradient regularization is integrated with the low rank regularization into the low rank low gradient approach for depth image inpainting. We compare our proposed low gradient regularization with sparse gradient regularization. The experimental results show the effectiveness of our proposed approach.