MIPI 2022 Challenge on Quad-Bayer Re-mosaic: Dataset and Report

TL;DR

This paper introduces the first MIPI challenge focusing on Quad-Bayer image re-mosaic and denoising, providing a new dataset and evaluating various models for Quad CFA to Bayer interpolation at full resolution.

Contribution

It presents a new dataset, challenge tracks, and comprehensive evaluation for Quad-Bayer re-mosaic and denoise, advancing research in mobile computational photography.

Findings

Multiple models evaluated with objective metrics

Dataset includes high-quality Quad and Bayer pairs under various noise levels

Benchmark results provided for future research

Abstract

Developing and integrating advanced image sensors with novel algorithms in camera systems are prevalent with the increasing demand for computational photography and imaging on mobile platforms. However, the lack of high-quality data for research and the rare opportunity for in-depth exchange of views from industry and academia constrain the development of mobile intelligent photography and imaging (MIPI). To bridge the gap, we introduce the first MIPI challenge, including five tracks focusing on novel image sensors and imaging algorithms. In this paper, Quad Joint Remosaic and Denoise, one of the five tracks, working on the interpolation of Quad CFA to Bayer at full resolution, is introduced. The participants were provided a new dataset, including 70 (training) and 15 (validation) scenes of high-quality Quad and Bayer pairs. In addition, for each scene, Quad of different noise levels was provided at 0dB, 24dB, and 42dB. All the data were captured using a Quad sensor in both outdoor and indoor conditions. The final results are evaluated using objective metrics, including PSNR, SSIM, LPIPS, and KLD. A detailed description of all models developed in this challenge is provided in this paper. More details of this challenge and the link to the dataset can be found at https://github.com/mipi-challenge/MIPI2022.