Understanding and Tackling Scattering and Reflective Flare for Mobile Camera Systems

TL;DR

This paper introduces a comprehensive raw image dataset for mobile cameras to improve flare removal techniques, identifies ISP operations affecting flare mitigation, and provides insights for optimizing mobile camera image processing.

Contribution

It presents a new large-scale raw image dataset for mobile flare study and analyzes how ISP operations influence flare removal effectiveness.

Findings

ISP denoising, compression, sharpening impact flare removal

Dataset enables better generalization of flare removal algorithms

Insights into optimizing ISP for improved flare mitigation

Abstract

The rise of mobile devices has spurred advancements in camera technology and image quality. However, mobile photography still faces issues like scattering and reflective flares. While previous research has acknowledged the negative impact of the mobile devices' internal image signal processing pipeline (ISP) on image quality, the specific ISP operations that hinder flare removal have not been fully identified. In addition, current solutions only partially address ISP-related deterioration due to a lack of comprehensive raw image datasets for flare study. To bridge these research gaps, we introduce a new raw image dataset tailored for mobile camera systems, focusing on eliminating flare. This dataset encompasses over 2,000 high-quality, full-resolution raw image pairs for scattering flare, and 1,200 for reflective flare, captured across various real-world scenarios, mobile devices, and camera settings. It is designed to enhance the generalizability of flare removal algorithms across a wide spectrum of conditions. Through detailed experiments, we have identified that ISP operations, such as denoising, compression, and sharpening, may either improve or obstruct flare removal, offering critical insights into optimizing ISP configurations for better flare mitigation. Our dataset is poised to advance the understanding of flare-related challenges, enabling more precise incorporation of flare removal steps into the ISP.