Beyond Calibration: Physically Informed Learning for Raw-to-Raw Mapping

TL;DR

This paper introduces the Neural Physical Model (NPM), a physically-informed learning approach for raw-to-raw camera mapping that adapts to illumination changes and outperforms existing methods in color consistency tasks.

Contribution

The paper presents NPM, a novel lightweight model that incorporates physical principles to improve raw-to-raw mapping across different cameras and illumination conditions.

Findings

NPM outperforms state-of-the-art methods on public datasets.

NPM adapts effectively to varying illumination conditions.

NPM supports training with or without paired data.

Abstract

Achieving consistent color reproduction across multiple cameras is essential for seamless image fusion and Image Processing Pipeline (ISP) compatibility in modern devices, but it is a challenging task due to variations in sensors and optics. Existing raw-to-raw conversion methods face limitations such as poor adaptability to changing illumination, high computational costs, or impractical requirements such as simultaneous camera operation and overlapping fields-of-view. We introduce the Neural Physical Model (NPM), a lightweight, physically-informed approach that simulates raw images under specified illumination to estimate transformations between devices. The NPM effectively adapts to varying illumination conditions, can be initialized with physical measurements, and supports training with or without paired data. Experiments on public datasets like NUS and BeyondRGB demonstrate that NPM outperforms recent state-of-the-art methods, providing robust chromatic consistency across different sensors and optical systems.