Video Interpolation using Optical Flow and Laplacian Smoothness

TL;DR

This paper introduces a novel optical flow method for non-rigid video interpolation that incorporates Laplacian Cotangent Mesh constraints to improve local smoothness and deformation accuracy.

Contribution

It integrates Laplacian Mesh constraints into optical flow optimization, enhancing local smoothness in non-rigid video interpolation and enabling applications in 3D facial modeling.

Findings

Effective on benchmark datasets like Middlebury and Garg et al.

Improves local smoothness and deformation accuracy in video interpolation.

Applicable to 3D Morphable Facial Model construction.

Abstract

Non-rigid video interpolation is a common computer vision task. In this paper we present an optical flow approach which adopts a Laplacian Cotangent Mesh constraint to enhance the local smoothness. Similar to Li et al., our approach adopts a mesh to the image with a resolution up to one vertex per pixel and uses angle constraints to ensure sensible local deformations between image pairs. The Laplacian Mesh constraints are expressed wholly inside the optical flow optimization, and can be applied in a straightforward manner to a wide range of image tracking and registration problems. We evaluate our approach by testing on several benchmark datasets, including the Middlebury and Garg et al. datasets. In addition, we show application of our method for constructing 3D Morphable Facial Models from dynamic 3D data.