A Restoration Network as an Implicit Prior
Yuyang Hu, Mauricio Delbracio, Peyman Milanfar, Ulugbek S. Kamilov
TL;DR
This paper introduces a novel approach that leverages pre-trained image restoration networks as implicit priors for solving various inverse imaging problems, demonstrating state-of-the-art results.
Contribution
It generalizes the use of image denoisers as priors by allowing any restoration network to be employed, with a theoretical convergence analysis and practical superior performance.
Findings
Achieves state-of-the-art results in super-resolution tasks.
Provides a theoretical analysis of convergence to a stationary point.
Demonstrates broad applicability to inverse imaging problems.
Abstract
Image denoisers have been shown to be powerful priors for solving inverse problems in imaging. In this work, we introduce a generalization of these methods that allows any image restoration network to be used as an implicit prior. The proposed method uses priors specified by deep neural networks pre-trained as general restoration operators. The method provides a principled approach for adapting state-of-the-art restoration models for other inverse problems. Our theoretical result analyzes its convergence to a stationary point of a global functional associated with the restoration operator. Numerical results show that the method using a super-resolution prior achieves state-of-the-art performance both quantitatively and qualitatively. Overall, this work offers a step forward for solving inverse problems by enabling the use of powerful pre-trained restoration models as priors.
Peer Reviews
Decision·ICLR 2024 poster
Originality: The main contribution of the work is extending the formulation of plug and play reconstruction methods to utilize implicit priors other than only denoisers. Clarity: The theoretical analysis of their proposed idea is clearly presented. Significance: While the performance numbers seem not necessarily significant in terms of improvements, it could lead to more improvements in subsequent works.
1) The method utilizes a pre-trained SwinIR [1]. However, according to the Table 2 from SwinIR, the performance of SwinIR for the task super-resolution is ~6db higher in terms of PSNR on set5, which is confusing. The authors should include the SwinIR for all of the test datasets as a baseline and explain the reason there is a performance drop after adapting their method. 2) There is another main weakness in the experimental results. There seems to be no discussion on the sources of randomness f
As I mentioned above, the extension proposed in this paper is simple, although non-trivial, and the paper provides theoretical convergence guarantees and experimental validation. It is a solid paper and the authors show a clear and solid knowledge of the field and of the state of the art, which is comprehensively reviewed. In summary, although certainly not a breakthrough or a very exciting new method, it is a good quality piece of work.
Although this is a good quality paper, there are a few aspects that could be improved, some important, others less so. First, and most importantly, the paper lacks some discussion/analysis of why restoration networks for problems other than denoising can lead to better results than denoising regularizers. Do they learn better image priors? This is somewhat surprising, in that these other networks are expected to have learned (in addition to a "prior") also specific aspects of the particular pr
1. The paper is well written, and easy to follow. 2. It provides sound theory proof on the convergence of DRP.
1. As for evaluation metric, more commonly-used metrics should be employed to have a comprehensive comparison, such as SSIM, LPIPS. 2. More comparative methods should be considered, including the DIP prior-based methods DIPFKP (CVPR 2021) and BSRDM (CVPR 2022) and some directly trained based method, such as BSRNet (ICCV 2021), RealESRNet (ICCV 2021 workshop). 3. This is a non-blind method. The experiments only verify its simple case with known degradation. I wonder that is it able to handle the
Videos
Taxonomy
TopicsPhotoacoustic and Ultrasonic Imaging · Image and Signal Denoising Methods · Sparse and Compressive Sensing Techniques