Leveraging Multispectral Sensors for Color Correction in Mobile Cameras

TL;DR

This paper introduces a unified, learning-based approach for end-to-end color correction in mobile cameras that effectively combines high-resolution RGB and low-resolution multispectral data, significantly improving color accuracy.

Contribution

The paper presents a novel integrated framework that jointly leverages RGB and multispectral data for color correction, refactoring existing architectures and creating a new dataset for training and evaluation.

Findings

Reduces color correction error by up to 50% compared to baselines.

Demonstrates flexibility by adapting different image-to-image architectures.

Improves color accuracy and stability in mobile imaging applications.

Abstract

Recent advances in snapshot multispectral (MS) imaging have enabled compact, low-cost spectral sensors for consumer and mobile devices. By capturing richer spectral information than conventional RGB sensors, these systems can enhance key imaging tasks, including color correction. However, most existing methods treat the color correction pipeline in separate stages, often discarding MS data early in the process. We propose a unified, learning-based framework that performs end-to-end color correction and jointly leverages data from a high-resolution RGB sensor and an auxiliary low-resolution MS sensor. Our approach integrates the full pipeline within a single model, producing coherent and color-accurate outputs. We demonstrate the flexibility and generality of our framework by refactoring two different state-of-the-art image-to-image architectures. To support training and evaluation, we construct a dedicated dataset by aggregating and repurposing publicly available spectral datasets, rendering under multiple RGB camera sensitivities. Extensive experiments show that our approach improves color accuracy and stability, reducing error by up to 50% compared to RGB-only and MS-driven baselines. Code, models and dataset available at: https://lucacogo.github.io/Mobile-Spectral-CC/.