H-VFI: Hierarchical Frame Interpolation for Videos with Large Motions
Changlin Li, Guangyang Wu, Yanan Sun, Xin Tao, Chi-Keung Tang, Yu-Wing, Tai

TL;DR
H-VFI introduces a hierarchical transformer-based approach for video frame interpolation that effectively handles large motions by progressively refining deformable kernels across multiple scales, leading to superior results.
Contribution
The paper presents H-VFI, a novel hierarchical transformer method for large-motion video frame interpolation, and introduces the YouTube200K dataset for high-quality training and evaluation.
Findings
H-VFI outperforms state-of-the-art methods on multiple benchmarks.
The hierarchical approach effectively decomposes large motions into simpler sub-tasks.
The YouTube200K dataset enhances training for high-resolution, high-frame-rate videos.
Abstract
Capitalizing on the rapid development of neural networks, recent video frame interpolation (VFI) methods have achieved notable improvements. However, they still fall short for real-world videos containing large motions. Complex deformation and/or occlusion caused by large motions make it an extremely difficult problem in video frame interpolation. In this paper, we propose a simple yet effective solution, H-VFI, to deal with large motions in video frame interpolation. H-VFI contributes a hierarchical video interpolation transformer (HVIT) to learn a deformable kernel in a coarse-to-fine strategy in multiple scales. The learnt deformable kernel is then utilized in convolving the input frames for predicting the interpolated frame. Starting from the smallest scale, H-VFI updates the deformable kernel by a residual in succession based on former predicted kernels, intermediate interpolated…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Vision and Imaging · Advanced Image Processing Techniques · Image Processing Techniques and Applications
MethodsConvolution · Deformable Kernel
