RAW-Domain Degradation Models for Realistic Smartphone Super-Resolution

TL;DR

This paper introduces a method for improving smartphone super-resolution by modeling device-specific RAW image degradations through calibration and unprocessing, resulting in better real-world performance.

Contribution

It presents a principled degradation modeling approach that leverages calibration and unprocessing to generate realistic training data for RAW-to-RGB super-resolution.

Findings

Degradation modeling improves SR performance on real data.

Our SR model outperforms baselines trained on generic degradations.

Device-specific calibration enhances realism of synthetic training data.

Abstract

Digital zoom on smartphones relies on learning-based super-resolution (SR) models that operate on RAW sensor images, but obtaining sensor-specific training data is challenging due to the lack of ground-truth images. Synthetic data generation via ``unprocessing'' pipelines offers a potential solution by simulating the degradations that transform high-resolution (HR) images into their low-resolution (LR) counterparts. However, these pipelines can introduce domain gaps due to incomplete or unrealistic degradation modeling. In this paper, we demonstrate that principled and carefully designed degradation modeling can enhance SR performance in real-world conditions. Instead of relying on generic priors for camera blur and noise, we model device-specific degradations through calibration and unprocess publicly available rendered images into the RAW domain of different smartphones. Using these image pairs, we train a single-image RAW-to-RGB SR model and evaluate it on real data from a held-out device. Our experiments show that accurate degradation modeling leads to noticeable improvements, with our SR model outperforming baselines trained on large pools of arbitrarily chosen degradations.