A Subjective Quality Study for Video Frame Interpolation

TL;DR

This study presents a new subjective quality assessment for video frame interpolation using a specialized dataset, revealing that existing metrics poorly correlate with human perception and highlighting the need for new perceptual quality metrics.

Contribution

Introduces BVI-VFI, a comprehensive dataset for VFI quality assessment, and evaluates existing metrics, demonstrating their inadequacy in predicting perceived quality.

Findings

Existing metrics show poor correlation with human perception (SROCC < 0.6).

The BVI-VFI dataset is publicly available for future research.

Current metrics are insufficient for assessing VFI quality.

Abstract

Video frame interpolation (VFI) is one of the fundamental research areas in video processing and there has been extensive research on novel and enhanced interpolation algorithms. The same is not true for quality assessment of the interpolated content. In this paper, we describe a subjective quality study for VFI based on a newly developed video database, BVI-VFI. BVI-VFI contains 36 reference sequences at three different frame rates and 180 distorted videos generated using five conventional and learning based VFI algorithms. Subjective opinion scores have been collected from 60 human participants, and then employed to evaluate eight popular quality metrics, including PSNR, SSIM and LPIPS which are all commonly used for assessing VFI methods. The results indicate that none of these metrics provide acceptable correlation with the perceived quality on interpolated content, with the best-performing metric, LPIPS, offering a SROCC value below 0.6. Our findings show that there is an urgent need to develop a bespoke perceptual quality metric for VFI. The BVI-VFI dataset is publicly available and can be accessed at https://danier97.github.io/BVI-VFI/.