RMFA-Net: A Neural ISP for Real RAW to RGB Image Reconstruction

TL;DR

RMFA-Net is a novel neural ISP that improves raw to RGB image reconstruction by addressing raw data characteristics, black level correction, and uneven exposure, achieving state-of-the-art performance.

Contribution

The paper introduces RMFA-Net with implicit black level correction, a Three-Channel-Split mode, and a Retinex-based tone mapping module, advancing raw2rgb reconstruction techniques.

Findings

Achieves over 25 dB PSNR, surpassing previous methods by +1 dB.

Lightweight RMFANet-tiny maintains strong performance, surpassing SOTA by +0.5 dB.

Outperforms existing algorithms on Mobile AI 2022 dataset.

Abstract

Deep learning-based ISP algorithms have demonstrated significant potential in raw2rgb reconstruction. However, existing networks have not fully considered the specific characteristics of raw data, such as black level and CFA, which can negatively impact texture and color if mishandled. Moreover, uneven exposure in raw data is also not considered carefully, leading to adverse effects on contrast and brightness. In this paper, we introduce RMFA-Net to tackle these problems. We perform implicit black level correction to mitigate color shifts in dim scenes. To preserve high-frequency information and prevent misalignment, we propose a novel Three-Channel-Split mode. To address the issue of uneven exposure, we designed an explicit tone mapping module based on the Retinex theory. We train and evaluate our models using the dataset released by the Mobile AI 2022 Learned Smartphone ISP Challenge. It is demonstrated that RMFA-Net outperforms previous algorithms, achieving a PSNR score of over 25 dB, surpassing the state-of-the-art by +1 dB. Furthermore, we developed a lightweight version, RMFANet-tiny, for engineering deployment while still maintaining strong performance, surpassing the SOTA by +0.5 dB.