Enhancing RAW-to-sRGB with Decoupled Style Structure in Fourier Domain

TL;DR

FourierISP introduces a frequency domain approach that decouples style and structure for RAW-to-sRGB conversion, achieving state-of-the-art results by independently optimizing color and spatial details.

Contribution

The paper proposes a novel Neural ISP framework that separates image restoration and enhancement in the frequency domain, improving RAW-to-sRGB mapping performance.

Findings

Achieves state-of-the-art results across multiple datasets.

Effectively refines image structure and color separately.

Demonstrates the benefit of frequency domain decoupling in image processing.

Abstract

RAW to sRGB mapping, which aims to convert RAW images from smartphones into RGB form equivalent to that of Digital Single-Lens Reflex (DSLR) cameras, has become an important area of research. However, current methods often ignore the difference between cell phone RAW images and DSLR camera RGB images, a difference that goes beyond the color matrix and extends to spatial structure due to resolution variations. Recent methods directly rebuild color mapping and spatial structure via shared deep representation, limiting optimal performance. Inspired by Image Signal Processing (ISP) pipeline, which distinguishes image restoration and enhancement, we present a novel Neural ISP framework, named FourierISP. This approach breaks the image down into style and structure within the frequency domain, allowing for independent optimization. FourierISP is comprised of three subnetworks: Phase Enhance Subnet for structural refinement, Amplitude Refine Subnet for color learning, and Color Adaptation Subnet for blending them in a smooth manner. This approach sharpens both color and structure, and extensive evaluations across varied datasets confirm that our approach realizes state-of-the-art results. Code will be available at ~\url{https://github.com/alexhe101/FourierISP}.