Benchmarking Video Frame Interpolation

TL;DR

This paper introduces a comprehensive benchmarking framework for video frame interpolation, addressing inconsistencies in evaluation metrics, dataset biases, and computational efficiency to enable fairer and more thorough comparisons of methods.

Contribution

It proposes a standardized benchmark with consistent error metrics, synthetic test data adhering to linearity assumptions, and an analysis of interpolation quality across different attributes.

Findings

Establishes a unified evaluation platform for frame interpolation methods.

Provides insights into how various factors affect interpolation quality.

Evaluates computational efficiency of different approaches.

Abstract

Video frame interpolation, the task of synthesizing new frames in between two or more given ones, is becoming an increasingly popular research target. However, the current evaluation of frame interpolation techniques is not ideal. Due to the plethora of test datasets available and inconsistent computation of error metrics, a coherent and fair comparison across papers is very challenging. Furthermore, new test sets have been proposed as part of method papers so they are unable to provide the in-depth evaluation of a dedicated benchmarking paper. Another severe downside is that these test sets violate the assumption of linearity when given two input frames, making it impossible to solve without an oracle. We hence strongly believe that the community would greatly benefit from a benchmarking paper, which is what we propose. Specifically, we present a benchmark which establishes consistent error metrics by utilizing a submission website that computes them, provides insights by analyzing the interpolation quality with respect to various per-pixel attributes such as the motion magnitude, contains a carefully designed test set adhering to the assumption of linearity by utilizing synthetic data, and evaluates the computational efficiency in a coherent manner.