Flow Guidance Deformable Compensation Network for Video Frame Interpolation

TL;DR

This paper introduces FGDCN, a novel video frame interpolation method that combines flow estimation and deformable compensation to improve accuracy, especially in challenging large motion and occlusion scenarios.

Contribution

The paper proposes a two-step process with flow guidance and deformable compensation, enhancing VFI accuracy and efficiency over existing methods.

Findings

Achieves superior performance on multiple datasets.

Uses fewer parameters than comparable methods.

Effectively handles large motion and occlusion.

Abstract

Motion-based video frame interpolation (VFI) methods have made remarkable progress with the development of deep convolutional networks over the past years. While their performance is often jeopardized by the inaccuracy of flow map estimation, especially in the case of large motion and occlusion. In this paper, we propose a flow guidance deformable compensation network (FGDCN) to overcome the drawbacks of existing motion-based methods. FGDCN decomposes the frame sampling process into two steps: a flow step and a deformation step. Specifically, the flow step utilizes a coarse-to-fine flow estimation network to directly estimate the intermediate flows and synthesizes an anchor frame simultaneously. To ensure the accuracy of the estimated flow, a distillation loss and a task-oriented loss are jointly employed in this step. Under the guidance of the flow priors learned in step one, the deformation step designs a pyramid deformable compensation network to compensate for the missing details of the flow step. In addition, a pyramid loss is proposed to supervise the model in both the image and frequency domain. Experimental results show that the proposed algorithm achieves excellent performance on various datasets with fewer parameters.