Reversing Image Signal Processors by Reverse Style Transferring

TL;DR

This paper introduces RST-ISP-Net, a novel neural network architecture that models the reversal of image signal processing operations as a style transfer problem, aiming to recover RAW images from sRGB images.

Contribution

The paper proposes a new end-to-end learning framework, RST-ISP-Net, that models ISP reversal as style transfer using adaptive feature normalization, and demonstrates its application in reversing ISP operations.

Findings

Modeling ISP reversal as style transfer is feasible.

The proposed method participated in the AIM Reversed ISP challenge.

Further improvements are needed for competitive performance.

Abstract

RAW image datasets are more suitable than the standard RGB image datasets for the ill-posed inverse problems in low-level vision, but not common in the literature. There are also a few studies to focus on mapping sRGB images to RAW format. Mapping from sRGB to RAW format could be a relevant domain for reverse style transferring since the task is an ill-posed reversing problem. In this study, we seek an answer to the question: Can the ISP operations be modeled as the style factor in an end-to-end learning pipeline? To investigate this idea, we propose a novel architecture, namely RST-ISP-Net, for learning to reverse the ISP operations with the help of adaptive feature normalization. We formulate this problem as a reverse style transferring and mostly follow the practice used in the prior work. We have participated in the AIM Reversed ISP challenge with our proposed architecture. Results indicate that the idea of modeling disruptive or modifying factors as style is still valid, but further improvements are required to be competitive in such a challenge.