Global-Local Stepwise Generative Network for Ultra High-Resolution Image Restoration

TL;DR

This paper introduces GLSGN, a novel stepwise network for ultra high-resolution image restoration that effectively combines local and global pathways, and presents a new dataset for reflection removal and rain streak removal.

Contribution

The paper proposes the first ultra high-resolution image restoration model with a multi-pathway approach and introduces a new dataset for reflection and rain streak removal tasks.

Findings

Outperforms state-of-the-art methods in multiple restoration tasks

Effectively combines local and global information for high-quality restoration

Provides a new dataset with 4,670 images for reflection and rain removal

Abstract

While the research on image background restoration from regular size of degraded images has achieved remarkable progress, restoring ultra high-resolution (e.g., 4K) images remains an extremely challenging task due to the explosion of computational complexity and memory usage, as well as the deficiency of annotated data. In this paper we present a novel model for ultra high-resolution image restoration, referred to as the Global-Local Stepwise Generative Network (GLSGN), which employs a stepwise restoring strategy involving four restoring pathways: three local pathways and one global pathway. The local pathways focus on conducting image restoration in a fine-grained manner over local but high-resolution image patches, while the global pathway performs image restoration coarsely on the scale-down but intact image to provide cues for the local pathways in a global view including semantics and noise patterns. To smooth the mutual collaboration between these four pathways, our GLSGN is designed to ensure the inter-pathway consistency in four aspects in terms of low-level content, perceptual attention, restoring intensity and high-level semantics, respectively. As another major contribution of this work, we also introduce the first ultra high-resolution dataset to date for both reflection removal and rain streak removal, comprising 4,670 real-world and synthetic images. Extensive experiments across three typical tasks for image background restoration, including image reflection removal, image rain streak removal and image dehazing, show that our GLSGN consistently outperforms state-of-the-art methods.