OmniZoomer: Learning to Move and Zoom in on Sphere at High-Resolution

TL;DR

OmniZoomer is a deep learning framework that effectively incorporates M"obius transformations for high-resolution, high-quality movement and zooming in on omnidirectional images, overcoming blurriness and aliasing issues.

Contribution

The paper introduces a novel deep learning approach integrating M"obius transformation with high-resolution feature enhancement and spherical resampling for improved ODIs.

Findings

Produces high-resolution, high-quality ODIs with flexible movement and zoom capabilities.

Alleviates blurriness caused by transformations in ODIs.

Reduces aliasing artifacts through spherical resampling.

Abstract

Omnidirectional images (ODIs) have become increasingly popular, as their large field-of-view (FoV) can offer viewers the chance to freely choose the view directions in immersive environments such as virtual reality. The M\"obius transformation is typically employed to further provide the opportunity for movement and zoom on ODIs, but applying it to the image level often results in blurry effect and aliasing problem. In this paper, we propose a novel deep learning-based approach, called \textbf{OmniZoomer}, to incorporate the M\"obius transformation into the network for movement and zoom on ODIs. By learning various transformed feature maps under different conditions, the network is enhanced to handle the increasing edge curvatures, which alleviates the blurry effect. Moreover, to address the aliasing problem, we propose two key components. Firstly, to compensate for the lack of pixels for describing curves, we enhance the feature maps in the high-resolution (HR) space and calculate the transformed index map with a spatial index generation module. Secondly, considering that ODIs are inherently represented in the spherical space, we propose a spherical resampling module that combines the index map and HR feature maps to transform the feature maps for better spherical correlation. The transformed feature maps are decoded to output a zoomed ODI. Experiments show that our method can produce HR and high-quality ODIs with the flexibility to move and zoom in to the object of interest. Project page is available at http://vlislab22.github.io/OmniZoomer/.