Overexposure Mask Fusion: Generalizable Reverse ISP Multi-Step Refinement

TL;DR

This paper introduces a novel multi-step refinement method with overexposure mask fusion for RAW image reconstruction, improving upon baseline models and offering a generalizable approach for enhancing high-performance image processing techniques.

Contribution

It presents a state-of-the-art RAW reconstruction pipeline that trains from RGB to demosaiced RAW, incorporating multi-step refinement and overexposure masks for improved accuracy and generalizability.

Findings

Enhanced RAW reconstruction performance over baseline U-Net

Effective multi-step refinement process improves results

Generalizable pipeline supports other end-to-end learning methods

Abstract

With the advent of deep learning methods replacing the ISP in transforming sensor RAW readings into RGB images, numerous methodologies solidified into real-life applications. Equally potent is the task of inverting this process which will have applications in enhancing computational photography tasks that are conducted in the RAW domain, addressing lack of available RAW data while reaping from the benefits of performing tasks directly on sensor readings. This paper's proposed methodology is a state-of-the-art solution to the task of RAW reconstruction, and the multi-step refinement process integrating an overexposure mask is novel in three ways: instead of from RGB to bayer, the pipeline trains from RGB to demosaiced RAW allowing use of perceptual loss functions; the multi-step processes has greatly enhanced the performance of the baseline U-Net from start to end; the pipeline is a generalizable process of refinement that can enhance other high performance methodologies that support end-to-end learning.