SAM-Deblur: Let Segment Anything Boost Image Deblurring

TL;DR

SAM-Deblur introduces a novel framework that leverages the Segment Anything Model to incorporate structural priors into image deblurring, significantly improving performance on multiple datasets.

Contribution

The paper presents the first integration of SAM into deblurring, proposing a mask dropout training method and a Mask Average Pooling unit to enhance deblurring models with structural priors.

Findings

Improved PSNR on RealBlurJ, ReloBlur, and REDS datasets.

Effective incorporation of SAM priors enhances deblurring quality.

The MAP unit is a versatile plug-and-play component.

Abstract

Image deblurring is a critical task in the field of image restoration, aiming to eliminate blurring artifacts. However, the challenge of addressing non-uniform blurring leads to an ill-posed problem, which limits the generalization performance of existing deblurring models. To solve the problem, we propose a framework SAM-Deblur, integrating prior knowledge from the Segment Anything Model (SAM) into the deblurring task for the first time. In particular, SAM-Deblur is divided into three stages. First, we preprocess the blurred images, obtain segment masks via SAM, and propose a mask dropout method for training to enhance model robustness. Then, to fully leverage the structural priors generated by SAM, we propose a Mask Average Pooling (MAP) unit specifically designed to average SAM-generated segmented areas, serving as a plug-and-play component which can be seamlessly integrated into existing deblurring networks. Finally, we feed the fused features generated by the MAP Unit into the deblurring model to obtain a sharp image. Experimental results on the RealBlurJ, ReloBlur, and REDS datasets reveal that incorporating our methods improves GoPro-trained NAFNet's PSNR by 0.05, 0.96, and 7.03, respectively. Project page is available at GitHub \href{https://hplqaq.github.io/projects/sam-deblur}{HPLQAQ/SAM-Deblur}.