Multi Kernel Estimation based Object Segmentation

TL;DR

This paper introduces Multi-KernelGAN, an advanced method that estimates multiple kernels for different image regions, improving super-resolution performance over traditional single-kernel approaches.

Contribution

It extends KernelGAN to estimate multiple kernels based on object segmentation, integrating deep learning models like YOLOv8 and SAM for enhanced accuracy.

Findings

Multi-kernel estimation outperforms single-kernel methods in super-resolution.

Combining YOLO and SAM yields the best kernel estimation results.

Experimental validation confirms improved image quality in super-resolution tasks.

Abstract

This paper presents a novel approach for multi-kernel estimation by enhancing the KernelGAN algorithm, which traditionally estimates a single kernel for the entire image. We introduce Multi-KernelGAN, which extends KernelGAN's capabilities by estimating two distinct kernels based on object segmentation masks. Our approach is validated through three distinct methods: texture-based patch Fast Fourier Transform (FFT) calculation, detail-based segmentation, and deep learning-based object segmentation using YOLOv8 and the Segment Anything Model (SAM). Among these methods, the combination of YOLO and SAM yields the best results for kernel estimation. Experimental results demonstrate that our multi-kernel estimation technique outperforms conventional single-kernel methods in super-resolution tasks.