Efficient motion-based metrics for video frame interpolation

TL;DR

This paper introduces a new motion divergence-based metric for evaluating video frame interpolation quality, which correlates reasonably with perceptual scores and is computationally more efficient than existing metrics.

Contribution

We propose a novel motion divergence metric for assessing frame interpolation quality that is both effective and computationally efficient.

Findings

The new metric correlates with perceptual scores (PLCC=0.51).

It offers a 2.7x speedup over FloLPIPS.

It favors perceptually pleasing interpolations over traditional metrics.

Abstract

Video frame interpolation (VFI) offers a way to generate intermediate frames between consecutive frames of a video sequence. Although the development of advanced frame interpolation algorithms has received increased attention in recent years, assessing the perceptual quality of interpolated content remains an ongoing area of research. In this paper, we investigate simple ways to process motion fields, with the purposes of using them as video quality metric for evaluating frame interpolation algorithms. We evaluate these quality metrics using the BVI-VFI dataset which contains perceptual scores measured for interpolated sequences. From our investigation we propose a motion metric based on measuring the divergence of motion fields. This metric correlates reasonably with these perceptual scores (PLCC=0.51) and is more computationally efficient (x2.7 speedup) compared to FloLPIPS (a well known motion-based metric). We then use our new proposed metrics to evaluate a range of state of the art frame interpolation metrics and find our metrics tend to favour more perceptual pleasing interpolated frames that may not score highly in terms of PSNR or SSIM.