# Continuous Space-Time Video Super-Resolution Utilizing Long-Range Temporal Information

## TL;DR

This paper introduces a continuous space-time video super-resolution method that can upscale videos to arbitrary frame rates and resolutions by leveraging long-range temporal information and novel modules for flexible, high-quality reconstruction.

## Contribution

The proposed C-STVSR method enables arbitrary frame rate and resolution conversion, utilizing long-range temporal data and innovative modules for improved video super-resolution performance.

## Key findings

- Outperforms state-of-the-art methods in objective evaluations.
- Achieves better visual quality in reconstructed videos.
- Demonstrates flexibility in various datasets.

## Abstract

In this paper, we consider the task of space-time video super-resolution (ST-VSR), namely, expanding a given source video to a higher frame rate and resolution simultaneously. However, most existing schemes either consider a fixed intermediate time and scale in the training stage or only accept a preset number of input frames (e.g., two adjacent frames) that fails to exploit long-range temporal information. To address these problems, we propose a continuous ST-VSR (C-STVSR) method that can convert the given video to any frame rate and spatial resolution. To achieve time-arbitrary interpolation, we propose a forward warping guided frame synthesis module and an optical-flow-guided context consistency loss to better approximate extreme motion and preserve similar structures among input and prediction frames. In addition, we design a memory-friendly cascading depth-to-space module to realize continuous spatial upsampling. Meanwhile, with the sophisticated reorganization of optical flow, the proposed method is memory friendly, making it possible to propagate information from long-range neighboring frames and achieve better reconstruction quality. Extensive experiments show that the proposed algorithm has good flexibility and achieves better performance on various datasets compared with the state-of-the-art methods in both objective evaluations and subjective visual effects.