Shape from Blur: Recovering Textured 3D Shape and Motion of Fast Moving Objects

TL;DR

This paper introduces a method to jointly reconstruct the 3D shape, texture, and motion of fast-moving objects from a single blurred image, improving deblurring and 3D reconstruction accuracy.

Contribution

It presents a novel 3D domain modeling approach for motion deblurring that outperforms existing 2D-based methods and enables high-fidelity textured 3D mesh recovery.

Findings

Outperforms competing methods on multiple benchmarks

Produces high-quality 3D meshes with detailed textures

Enables temporal super-resolution and novel view synthesis

Abstract

We address the novel task of jointly reconstructing the 3D shape, texture, and motion of an object from a single motion-blurred image. While previous approaches address the deblurring problem only in the 2D image domain, our proposed rigorous modeling of all object properties in the 3D domain enables the correct description of arbitrary object motion. This leads to significantly better image decomposition and sharper deblurring results. We model the observed appearance of a motion-blurred object as a combination of the background and a 3D object with constant translation and rotation. Our method minimizes a loss on reconstructing the input image via differentiable rendering with suitable regularizers. This enables estimating the textured 3D mesh of the blurred object with high fidelity. Our method substantially outperforms competing approaches on several benchmarks for fast moving objects deblurring. Qualitative results show that the reconstructed 3D mesh generates high-quality temporal super-resolution and novel views of the deblurred object.