MIPI 2022 Challenge on RGBW Sensor Re-mosaic: Dataset and Report

TL;DR

This paper introduces a new dataset and challenge focused on RGBW sensor re-mosaic and denoising, providing a benchmark for advancing image sensor algorithms in mobile photography.

Contribution

It presents the first MIPI challenge on RGBW re-mosaic, including a high-quality dataset and evaluation framework for sensor interpolation and denoising algorithms.

Findings

Participants developed diverse models for RGBW to Bayer interpolation.

The challenge results highlight the effectiveness of different algorithms based on objective metrics.

The dataset enables standardized benchmarking for RGBW sensor processing.

Abstract

Developing and integrating advanced image sensors with novel algorithms in camera systems are prevalent with the increasing demand for computational photography and imaging on mobile platforms. However, the lack of high-quality data for research and the rare opportunity for in-depth exchange of views from industry and academia constrain the development of mobile intelligent photography and imaging (MIPI). To bridge the gap, we introduce the first MIPI challenge including five tracks focusing on novel image sensors and imaging algorithms. In this paper, RGBW Joint Remosaic and Denoise, one of the five tracks, working on the interpolation of RGBW CFA to Bayer at full resolution, is introduced. The participants were provided with a new dataset including 70 (training) and 15 (validation) scenes of high-quality RGBW and Bayer pairs. In addition, for each scene, RGBW of different noise levels was provided at 0dB, 24dB, and 42dB. All the data were captured using an RGBW sensor in both outdoor and indoor conditions. The final results are evaluated using objective metrics including PSNR, SSIM, LPIPS, and KLD. A detailed description of all models developed in this challenge is provided in this paper. More details of this challenge and the link to the dataset can be found at https://github.com/mipi-challenge/MIPI2022.