AceVFI: A Comprehensive Survey of Advances in Video Frame Interpolation

TL;DR

AceVFI provides a comprehensive review of over 250 papers on Video Frame Interpolation, categorizing methods, analyzing challenges, and discussing future directions in this evolving field.

Contribution

This paper offers the first extensive survey covering recent advances in VFI, including diverse methodologies, classifications, and application insights.

Findings

Deep learning methods dominate VFI advancements

Key challenges include large motion and occlusion

Future research directions identified

Abstract

Video Frame Interpolation (VFI) is a core low-level vision task that synthesizes intermediate frames between existing ones while ensuring spatial and temporal coherence. Over the past decades, VFI methodologies have evolved from classical motion compensation-based approach to a wide spectrum of deep learning-based approaches, including kernel-, flow-, hybrid-, phase-, GAN-, Transformer-, Mamba-, and most recently, diffusion-based models. We introduce AceVFI, a comprehensive and up-to-date review of the VFI field, covering over 250 representative papers. We systematically categorize VFI methods based on their core design principles and architectural characteristics. Further, we classify them into two major learning paradigms: Center-Time Frame Interpolation (CTFI) and Arbitrary-Time Frame Interpolation (ATFI). We analyze key challenges in VFI, including large motion, occlusion, lighting variation, and non-linear motion. In addition, we review standard datasets, loss functions, evaluation metrics. We also explore VFI applications in other domains and highlight future research directions. This survey aims to serve as a valuable reference for researchers and practitioners seeking a thorough understanding of the modern VFI landscape.