Conformal Bounds on Full-Reference Image Quality for Imaging Inverse Problems
Jeffrey Wen, Rizwan Ahmad, and Philip Schniter
TL;DR
This paper introduces a method combining conformal prediction with approximate posterior sampling to provide guaranteed bounds on full-reference image quality metrics in inverse imaging problems, crucial for safety-critical applications.
Contribution
It presents a novel approach to quantify uncertainty in image quality assessment for inverse problems using conformal bounds, applicable to medical imaging and denoising.
Findings
Provides guaranteed bounds on FRIQ metrics like SSIM and PSNR.
Demonstrates effectiveness on MRI and denoising tasks.
Offers a practical tool for safety-critical imaging applications.
Abstract
In imaging inverse problems, we would like to know how close the recovered image is to the true image in terms of full-reference image quality (FRIQ) metrics like PSNR, SSIM, LPIPS, etc. This is especially important in safety-critical applications like medical imaging, where knowing that, say, the SSIM was poor could potentially avoid a costly misdiagnosis. But since we don't know the true image, computing FRIQ is non-trivial. In this work, we combine conformal prediction with approximate posterior sampling to construct bounds on FRIQ that are guaranteed to hold up to a user-specified error probability. We demonstrate our approach on image denoising and accelerated magnetic resonance imaging (MRI) problems. Code is available at https://github.com/jwen307/quality_uq.
Peer Reviews
Decision·Submitted to ICLR 2025
1, The research question is significant from various perspectives, especially for trustworthy machine learning, as it addresses the need for reliable uncertainty quantification in image reconstruction tasks. 2, Another strength of this paper’s novelty is its integration of conformal prediction with approximate posterior sampling to construct statistically rigorous bounds on FRIQ metrics for imaging inverse problems, offering guaranteed coverage with a user-specified error probability. This appr
1, The paper in its current form lacks a more comprehensive review of other uncertainty quantification methods, such as Bayesian approaches (e.g., Monte Carlo dropout), which are widely used in imaging reconstruction. Including these comparisons would better highlight the proposed method’s advantages in terms of coverage guarantees and reliability. 2, Similarly, the paper also lacks numerical comparisons with other uncertainty quantification methods beyond conformal prediction. Including these
**Novelty and potential real-life application** * The paper utilizes conformal prediction (CP) , which is well-suited for generating uncertainty bounds, to calculate bounds on full-reference image quality metrics. This can be particularly useful in medical imaging, especially in scenarios like the multi-round MRI acquisition described in lines $463-464$. * Multiple bounds, including an adaptive bound and its improved version, are provided, and numerical comparisons between each bound are made.
**Most of the weaknesses are identified and discussed in the Limitations paragraph (Line $509$)** * As stated, the method may not work if the calibration and test data distributions are significantly different. I acknowledge it requires more work to make the method more robust to distribution shift, but a simple experiment demonstrating the sensitivity to the such shifts can be added. * The performance difference between the adaptive bound and the improved adaptive bound appears incremental in
1. The considered problem is novel and of sufficient interest to the computational imaging community. 2. The application of conformal prediction to imaging is also relatively new. 3. The current manuscript properly discusses the limitations of the proposed method.
1. The clarity of the method section needs to be improved. 2. The mathematical notations are abused, which hinders first-time readers to quickly understand the method. 3. The above two points jointly affect proper interpretation of the experimental results.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Image Fusion Techniques · Image and Signal Denoising Methods · Infrared Target Detection Methodologies