Affine-modeled video extraction from a single motion blurred image

TL;DR

This paper introduces a generalized affine motion modeling approach for extracting sharp video frames from a single motion-blurred image, effectively handling complex motions like rotation and depth changes through segmentation and optimization.

Contribution

The work presents a novel affine-based video extraction method that models complex motions, segments objects, and employs a coarse-to-fine optimization strategy for improved accuracy.

Findings

Achieves state-of-the-art results on public datasets.

Effectively handles multiple complex motion types.

Reduces artifacts with $l0$-norm total variation regularization.

Abstract

A motion-blurred image is the temporal average of multiple sharp frames over the exposure time. Recovering these sharp video frames from a single blurred image is nontrivial, due to not only its strong ill-posedness, but also various types of complex motion in reality such as rotation and motion in depth. In this work, we report a generalized video extraction method using the affine motion modeling, enabling to tackle multiple types of complex motion and their mixing. In its workflow, the moving objects are first segemented in the alpha channel. This allows separate recovery of different objects with different motion. Then, we reduce the variable space by modeling each video clip as a series of affine transformations of a reference frame, and introduce the $l0$-norm total variation regularization to attenuate the ringing artifact. The differentiable affine operators are employed to realize gradient-descent optimization of the affine model, which follows a novel coarse-to-fine strategy to further reduce artifacts. As a result, both the affine parameters and sharp reference image are retrieved. They are finally input into stepwise affine transformation to recover the sharp video frames. The stepwise retrieval maintains the nature to bypass the frame order ambiguity. Experiments on both public datasets and real captured data validate the state-of-the-art performance of the reported technique.