OPDN: Omnidirectional Position-aware Deformable Network for Omnidirectional Image Super-Resolution

TL;DR

This paper introduces a two-stage super-resolution framework for 360-degree omnidirectional images, utilizing position-aware deformable blocks, Fourier upsampling, and frequency fusion to enhance image quality, winning the NTIRE 2023 challenge.

Contribution

The paper proposes a novel two-stage framework with omnidirectional position-aware deformable blocks and frequency fusion, specifically designed for 360-degree image super-resolution, outperforming existing methods.

Findings

Achieved superior super-resolution performance on 360-degree images.

Won the NTIRE 2023 challenge for omnidirectional image super-resolution.

Enhanced feature extraction and high-frequency detail preservation.

Abstract

360{\deg} omnidirectional images have gained research attention due to their immersive and interactive experience, particularly in AR/VR applications. However, they suffer from lower angular resolution due to being captured by fisheye lenses with the same sensor size for capturing planar images. To solve the above issues, we propose a two-stage framework for 360{\deg} omnidirectional image superresolution. The first stage employs two branches: model A, which incorporates omnidirectional position-aware deformable blocks (OPDB) and Fourier upsampling, and model B, which adds a spatial frequency fusion module (SFF) to model A. Model A aims to enhance the feature extraction ability of 360{\deg} image positional information, while Model B further focuses on the high-frequency information of 360{\deg} images. The second stage performs same-resolution enhancement based on the structure of model A with a pixel unshuffle operation. In addition, we collected data from YouTube to improve the fitting ability of the transformer, and created pseudo low-resolution images using a degradation network. Our proposed method achieves superior performance and wins the NTIRE 2023 challenge of 360{\deg} omnidirectional image super-resolution.