Joint Estimation of Camera Pose, Depth, Deblurring, and Super-Resolution from a Blurred Image Sequence

TL;DR

This paper introduces a unified framework that simultaneously estimates camera pose, depth, deblurring, and super-resolution from blurry image sequences, outperforming traditional sequential approaches.

Contribution

It presents the first integrated approach that jointly solves four interconnected problems by reflecting the physical imaging process within a unified optimization framework.

Findings

High-quality depth maps from severely degraded images

Outstanding deblurred and super-resolved images

Outperforms naive multi-view stereo methods

Abstract

The conventional methods for estimating camera poses and scene structures from severely blurry or low resolution images often result in failure. The off-the-shelf deblurring or super-resolution methods may show visually pleasing results. However, applying each technique independently before matching is generally unprofitable because this naive series of procedures ignores the consistency between images. In this paper, we propose a pioneering unified framework that solves four problems simultaneously, namely, dense depth reconstruction, camera pose estimation, super-resolution, and deblurring. By reflecting a physical imaging process, we formulate a cost minimization problem and solve it using an alternating optimization technique. The experimental results on both synthetic and real videos show high-quality depth maps derived from severely degraded images that contrast the failures of naive multi-view stereo methods. Our proposed method also produces outstanding deblurred and super-resolved images unlike the independent application or combination of conventional video deblurring, super-resolution methods.