Video Frame Interpolation Based on Deformable Kernel Region

TL;DR

This paper introduces a deformable kernel region approach for video frame interpolation, overcoming grid restrictions to better adapt to object shapes and motion uncertainties, leading to more accurate interpolated frames.

Contribution

It applies deformable convolution to video interpolation, allowing flexible kernel regions that improve accuracy over traditional fixed-grid methods.

Findings

Outperforms state-of-the-art methods on four datasets.

Better adapts to irregular object shapes and motion.

Achieves more accurate frame synthesis.

Abstract

Video frame interpolation task has recently become more and more prevalent in the computer vision field. At present, a number of researches based on deep learning have achieved great success. Most of them are either based on optical flow information, or interpolation kernel, or a combination of these two methods. However, these methods have ignored that there are grid restrictions on the position of kernel region during synthesizing each target pixel. These limitations result in that they cannot well adapt to the irregularity of object shape and uncertainty of motion, which may lead to irrelevant reference pixels used for interpolation. In order to solve this problem, we revisit the deformable convolution for video interpolation, which can break the fixed grid restrictions on the kernel region, making the distribution of reference points more suitable for the shape of the object, and thus warp a more accurate interpolation frame. Experiments are conducted on four datasets to demonstrate the superior performance of the proposed model in comparison to the state-of-the-art alternatives.