Learning for Unconstrained Space-Time Video Super-Resolution

TL;DR

This paper introduces an unconstrained space-time video super-resolution network that adaptively enhances both temporal and spatial resolution by leveraging space-time correlations, optical flow, and a generalized pixelshuffle, outperforming existing methods.

Contribution

It presents a flexible, efficient network capable of arbitrary space-time resolution enhancement, addressing limitations of prior methods that lacked adaptability and efficiency.

Findings

Outperforms state-of-the-art methods in quality

Uses fewer parameters and less computation

Provides flexible resolution adjustment

Abstract

Recent years have seen considerable research activities devoted to video enhancement that simultaneously increases temporal frame rate and spatial resolution. However, the existing methods either fail to explore the intrinsic relationship between temporal and spatial information or lack flexibility in the choice of final temporal/spatial resolution. In this work, we propose an unconstrained space-time video super-resolution network, which can effectively exploit space-time correlation to boost performance. Moreover, it has complete freedom in adjusting the temporal frame rate and spatial resolution through the use of the optical flow technique and a generalized pixelshuffle operation. Our extensive experiments demonstrate that the proposed method not only outperforms the state-of-the-art, but also requires far fewer parameters and less running time.