Real-Time Intermediate Flow Estimation for Video Frame Interpolation

TL;DR

RIFE is a real-time video frame interpolation method that uses a neural network for fast, high-quality intermediate flow estimation, outperforming existing methods in speed and accuracy.

Contribution

The paper introduces RIFE, a neural network-based VFI approach that achieves real-time performance without pre-trained models and supports arbitrary-timestep interpolation.

Findings

RIFE outperforms SuperSlomo and DAIN in speed and quality.

RIFE achieves state-of-the-art results on public benchmarks.

RIFE supports arbitrary-timestep frame interpolation with temporal encoding.

Abstract

Real-time video frame interpolation (VFI) is very useful in video processing, media players, and display devices. We propose RIFE, a Real-time Intermediate Flow Estimation algorithm for VFI. To realize a high-quality flow-based VFI method, RIFE uses a neural network named IFNet that can estimate the intermediate flows end-to-end with much faster speed. A privileged distillation scheme is designed for stable IFNet training and improve the overall performance. RIFE does not rely on pre-trained optical flow models and can support arbitrary-timestep frame interpolation with the temporal encoding input. Experiments demonstrate that RIFE achieves state-of-the-art performance on several public benchmarks. Compared with the popular SuperSlomo and DAIN methods, RIFE is 4--27 times faster and produces better results. Furthermore, RIFE can be extended to wider applications thanks to temporal encoding. The code is available at https://github.com/megvii-research/ECCV2022-RIFE.