VideoINR: Learning Video Implicit Neural Representation for Continuous Space-Time Super-Resolution

TL;DR

VideoINR introduces a novel implicit neural representation for videos that enables continuous space-time super-resolution, allowing arbitrary resolution and frame rate enhancement with competitive performance and superior out-of-distribution scaling.

Contribution

The paper proposes VideoINR, a continuous implicit neural representation for videos, enabling flexible super-resolution at arbitrary scales, unlike fixed-scale prior methods.

Findings

Achieves competitive results on standard up-sampling scales.

Outperforms prior methods on continuous and out-of-distribution scales.

Supports arbitrary spatial resolution and frame rate enhancement.

Abstract

Videos typically record the streaming and continuous visual data as discrete consecutive frames. Since the storage cost is expensive for videos of high fidelity, most of them are stored in a relatively low resolution and frame rate. Recent works of Space-Time Video Super-Resolution (STVSR) are developed to incorporate temporal interpolation and spatial super-resolution in a unified framework. However, most of them only support a fixed up-sampling scale, which limits their flexibility and applications. In this work, instead of following the discrete representations, we propose Video Implicit Neural Representation (VideoINR), and we show its applications for STVSR. The learned implicit neural representation can be decoded to videos of arbitrary spatial resolution and frame rate. We show that VideoINR achieves competitive performances with state-of-the-art STVSR methods on common up-sampling scales and significantly outperforms prior works on continuous and out-of-training-distribution scales. Our project page is at http://zeyuan-chen.com/VideoINR/ .