Learning Degradation-Independent Representations for Camera ISP Pipelines

TL;DR

This paper introduces a novel deep learning method that learns degradation-independent image representations to improve the robustness and generalization of camera ISP pipelines across various tasks.

Contribution

It proposes a new DNN approach for learning degradation-independent representations, enhancing generalization in image restoration and downstream tasks.

Findings

Outperforms state-of-the-art methods in blind image restoration

Improves object detection and segmentation under diverse degradations

Demonstrates strong domain generalization capability

Abstract

Image signal processing (ISP) pipeline plays a fundamental role in digital cameras, which converts raw Bayer sensor data to RGB images. However, ISP-generated images usually suffer from imperfections due to the compounded degradations that stem from sensor noises, demosaicing noises, compression artifacts, and possibly adverse effects of erroneous ISP hyperparameter settings such as ISO and gamma values. In a general sense, these ISP imperfections can be considered as degradations. The highly complex mechanisms of ISP degradations, some of which are even unknown, pose great challenges to the generalization capability of deep neural networks (DNN) for image restoration and to their adaptability to downstream tasks. To tackle the issues, we propose a novel DNN approach to learn degradation-independent representations (DiR) through the refinement of a self-supervised learned baseline representation. The proposed DiR learning technique has remarkable domain generalization capability and consequently, it outperforms state-of-the-art methods across various downstream tasks, including blind image restoration, object detection, and instance segmentation, as verified in our experiments.