Estimation of Camera Response Function using Prediction Consistency and Gradual Refinement with an Extension to Deep Learning

TL;DR

This paper introduces a non-deep-learning approach for estimating camera response functions from single images using prediction consistency and gradual refinement, and extends it with a deep learning method that performs test-time training for better generalization.

Contribution

The paper presents a novel non-deep-learning method for CRF estimation that is robust to noise and extends it with a deep learning approach utilizing test-time training for improved generalization.

Findings

Outperforms existing methods on real daytime and nighttime images.

The deep learning extension with test-time training improves generalization.

The gradual refinement scheme enhances robustness to noise.

Abstract

Most existing methods for CRF estimation from a single image fail to handle general real images. For instance, EdgeCRF based on colour patches extracted from edges works effectively only when the presence of noise is insignificant, which is not the case for many real images; and, CRFNet, a recent method based on fully supervised deep learning works only for the CRFs that are in the training data, and hence fail to deal with other possible CRFs beyond the training data. To address these problems, we introduce a non-deep-learning method using prediction consistency and gradual refinement. First, we rely more on the patches of the input image that provide more consistent predictions. If the predictions from a patch are more consistent, it means that the patch is likely to be less affected by noise or any inferior colour combinations, and hence, it can be more reliable for CRF estimation. Second, we employ a gradual refinement scheme in which we start from a simple CRF model to generate a result which is more robust to noise but less accurate, and then we gradually increase the model's complexity to improve the result. This is because a simple model, while being less accurate, overfits less to noise than a complex model does. Our experiments show that our method outperforms the existing single-image methods for daytime and nighttime real images. We further propose a more efficient deep learning extension that performs test-time training (based on unsupervised losses) on the test input image. This provides our method better generalization performance than CRFNet making it more practically applicable for CRF estimation for general real images.