A Systematic Investigation on Deep Learning-Based Omnidirectional Image and Video Super-Resolution

TL;DR

This paper systematically reviews deep learning methods for omnidirectional image and video super-resolution, introduces a new dataset with real-world distortions, and evaluates existing approaches to guide future research.

Contribution

It provides a comprehensive survey of recent deep learning techniques, introduces the authentic 360Insta dataset, and offers extensive evaluations to advance the field.

Findings

Existing datasets mainly use synthetic distortions.

Deep learning methods show varying generalization capabilities.

The new dataset enables more realistic evaluation.

Abstract

Omnidirectional image and video super-resolution is a crucial research topic in low-level vision, playing an essential role in virtual reality and augmented reality applications. Its goal is to reconstruct high-resolution images or video frames from low-resolution inputs, thereby enhancing detail preservation and enabling more accurate scene analysis and interpretation. In recent years, numerous innovative and effective approaches have been proposed, predominantly based on deep learning techniques, involving diverse network architectures, loss functions, projection strategies, and training datasets. This paper presents a systematic review of recent progress in omnidirectional image and video super-resolution, focusing on deep learning-based methods. Given that existing datasets predominantly rely on synthetic degradation and fall short in capturing real-world distortions, we introduce a new dataset, 360Insta, that comprises authentically degraded omnidirectional images and videos collected under diverse conditions, including varying lighting, motion, and exposure settings. This dataset addresses a critical gap in current omnidirectional benchmarks and enables more robust evaluation of the generalization capabilities of omnidirectional super-resolution methods. We conduct comprehensive qualitative and quantitative evaluations of existing methods on both public datasets and our proposed dataset. Furthermore, we provide a systematic overview of the current status of research and discuss promising directions for future exploration. All datasets, methods, and evaluation metrics introduced in this work are publicly available and will be regularly updated. Project page: https://github.com/nqian1/Survey-on-ODISR-and-ODVSR.