Implicit Neural Representation for Video Restoration

TL;DR

VR-INR introduces a flexible implicit neural representation method for video restoration that generalizes from a single training scale to arbitrary super-resolution scales and performs zero-shot denoising, outperforming existing methods.

Contribution

The paper presents VR-INR, a novel implicit neural representation framework trained on one scale that effectively generalizes to unseen scales and noise conditions in video restoration.

Findings

Outperforms state-of-the-art in sharpness and detail preservation.

Effectively generalizes to unseen super-resolution scales.

Successfully performs zero-shot denoising on noisy videos.

Abstract

High-resolution (HR) videos play a crucial role in many computer vision applications. Although existing video restoration (VR) methods can significantly enhance video quality by exploiting temporal information across video frames, they are typically trained for fixed upscaling factors and lack the flexibility to handle scales or degradations beyond their training distribution. In this paper, we introduce VR-INR, a novel video restoration approach based on Implicit Neural Representations (INRs) that is trained only on a single upscaling factor ($\times 4$) but generalizes effectively to arbitrary, unseen super-resolution scales at test time. Notably, VR-INR also performs zero-shot denoising on noisy input, despite never having seen noisy data during training. Our method employs a hierarchical spatial-temporal-texture encoding framework coupled with multi-resolution implicit hash encoding, enabling adaptive decoding of high-resolution and noise-suppressed frames from low-resolution inputs at any desired magnification. Experimental results show that VR-INR consistently maintains high-quality reconstructions at unseen scales and noise during training, significantly outperforming state-of-the-art approaches in sharpness, detail preservation, and denoising efficacy.