Temporal shape super-resolution by intra-frame motion encoding using high-fps structured light

TL;DR

This paper introduces a method for depth super-resolution in moving scenes by projecting multiple high-frequency patterns within a single image, enabling temporal reconstruction without geometric calibration.

Contribution

It proposes a novel intra-frame motion encoding technique using high-fps structured light and a learning-based decoding approach for depth reconstruction.

Findings

Effective reconstruction of sequential shapes from a single image

Outperforms recent techniques in quantitative evaluations

No geometric calibration required

Abstract

One of the solutions of depth imaging of moving scene is to project a static pattern on the object and use just a single image for reconstruction. However, if the motion of the object is too fast with respect to the exposure time of the image sensor, patterns on the captured image are blurred and reconstruction fails. In this paper, we impose multiple projection patterns into each single captured image to realize temporal super resolution of the depth image sequences. With our method, multiple patterns are projected onto the object with higher fps than possible with a camera. In this case, the observed pattern varies depending on the depth and motion of the object, so we can extract temporal information of the scene from each single image. The decoding process is realized using a learning-based approach where no geometric calibration is needed. Experiments confirm the effectiveness of our method where sequential shapes are reconstructed from a single image. Both quantitative evaluations and comparisons with recent techniques were also conducted.