On Variational Methods for Motion Compensated Inpainting

TL;DR

This paper introduces a Bayesian variational framework for joint motion estimation and inpainting in damaged video sequences, providing algorithms and experimental validation for synthetic and real data.

Contribution

It develops a generic Bayesian variational approach with multiresolution algorithms for simultaneous motion estimation and inpainting in videos, a novel integration of these tasks.

Findings

Effective inpainting and motion recovery demonstrated on synthetic sequences.

Multiresolution algorithms improve local minima convergence.

Experimental results validate the approach on real video data.

Abstract

We develop in this paper a generic Bayesian framework for the joint estimation of motion and recovery of missing data in a damaged video sequence. Using standard maximum a posteriori to variational formulation rationale, we derive generic minimum energy formulations for the estimation of a reconstructed sequence as well as motion recovery. We instantiate these energy formulations and from their Euler-Lagrange Equations, we propose a full multiresolution algorithms in order to compute good local minimizers for our energies and discuss their numerical implementations, focusing on the missing data recovery part, i.e. inpainting. Experimental results for synthetic as well as real sequences are presented. Image sequences and extra material is available at http://image.diku.dk/francois/seqinp.php.