3DAttGAN: A 3D Attention-based Generative Adversarial Network for Joint Space-Time Video Super-Resolution

TL;DR

This paper introduces 3DAttGAN, a novel 3D attention-based GAN that jointly enhances spatial resolution and frame rate in videos by effectively utilizing spatio-temporal information, leading to more realistic and detailed video super-resolution.

Contribution

The paper presents a new 3D attention mechanism within a GAN framework for joint space-time video super-resolution, improving upon existing methods by better exploiting spatio-temporal features.

Findings

Outperforms existing methods on Vid4, Vimeo-90K, and REDS datasets.

Effectively captures important spatial and temporal features with 3D attention.

Produces more accurate and realistic high-resolution, high-frame-rate videos.

Abstract

In many applications, including surveillance, entertainment, and restoration, there is a need to increase both the spatial resolution and the frame rate of a video sequence. The aim is to improve visual quality, refine details, and create a more realistic viewing experience. Existing space-time video super-resolution methods do not effectively use spatio-temporal information. To address this limitation, we propose a generative adversarial network for joint space-time video super-resolution. The generative network consists of three operations: shallow feature extraction, deep feature extraction, and reconstruction. It uses three-dimensional (3D) convolutions to process temporal and spatial information simultaneously and includes a novel 3D attention mechanism to extract the most important channel and spatial information. The discriminative network uses a two-branch structure to handle details and motion information, making the generated results more accurate. Experimental results on the Vid4, Vimeo-90K, and REDS datasets demonstrate the effectiveness of the proposed method. The source code is publicly available at https://github.com/FCongRui/3DAttGan.git.