FloLPIPS: A Bespoke Video Quality Metric for Frame Interpoation

TL;DR

FloLPIPS is a new video quality metric specifically designed for evaluating frame interpolation quality, outperforming existing assessors by incorporating temporal information and perceptual features.

Contribution

The paper introduces FloLPIPS, a novel full reference video quality metric tailored for VFI, enhancing LPIPS with temporal weighting based on optical flow to better assess interpolated videos.

Findings

FloLPIPS correlates more strongly with subjective quality scores than 12 existing metrics.

It demonstrates statistically significant improvements in evaluating interpolated video quality.

The method is validated on the BVI-VFI database with diverse artifacts.

Abstract

Video frame interpolation (VFI) serves as a useful tool for many video processing applications. Recently, it has also been applied in the video compression domain for enhancing both conventional video codecs and learning-based compression architectures. While there has been an increased focus on the development of enhanced frame interpolation algorithms in recent years, the perceptual quality assessment of interpolated content remains an open field of research. In this paper, we present a bespoke full reference video quality metric for VFI, FloLPIPS, that builds on the popular perceptual image quality metric, LPIPS, which captures the perceptual degradation in extracted image feature space. In order to enhance the performance of LPIPS for evaluating interpolated content, we re-designed its spatial feature aggregation step by using the temporal distortion (through comparing optical flows) to weight the feature difference maps. Evaluated on the BVI-VFI database, which contains 180 test sequences with various frame interpolation artefacts, FloLPIPS shows superior correlation performance (with statistical significance) with subjective ground truth over 12 popular quality assessors. To facilitate further research in VFI quality assessment, our code is publicly available at https://danier97.github.io/FloLPIPS.