BiM-VFI: Bidirectional Motion Field-Guided Frame Interpolation for Video with Non-uniform Motions

TL;DR

This paper introduces BiM-VFI, a novel video frame interpolation model that effectively handles non-uniform motions using a bidirectional motion description map, resulting in significantly clearer interpolated frames.

Contribution

The paper proposes a new motion description map, BiM, and a BiM-guided flow estimation method with knowledge distillation, advancing non-uniform motion handling in VFI.

Findings

Achieves 26% and 45% improvements in LPIPS and STLPIPS over state-of-the-art methods.

Produces interpolated frames with fewer blurs at arbitrary time instances.

Outperforms recent VFI models on multiple benchmarks.

Abstract

Existing Video Frame interpolation (VFI) models tend to suffer from time-to-location ambiguity when trained with video of non-uniform motions, such as accelerating, decelerating, and changing directions, which often yield blurred interpolated frames. In this paper, we propose (i) a novel motion description map, Bidirectional Motion field (BiM), to effectively describe non-uniform motions; (ii) a BiM-guided Flow Net (BiMFN) with Content-Aware Upsampling Network (CAUN) for precise optical flow estimation; and (iii) Knowledge Distillation for VFI-centric Flow supervision (KDVCF) to supervise the motion estimation of VFI model with VFI-centric teacher flows. The proposed VFI is called a Bidirectional Motion field-guided VFI (BiM-VFI) model. Extensive experiments show that our BiM-VFI model significantly surpasses the recent state-of-the-art VFI methods by 26% and 45% improvements in LPIPS and STLPIPS respectively, yielding interpolated frames with much fewer blurs at arbitrary time instances.