Localized Super Resolution for Foreground Images using U-Net and MR-CNN

TL;DR

This paper explores a localized super resolution approach for portrait images, emphasizing foreground enhancement by combining U-Net and MR-CNN, and evaluates its effectiveness using SSIM and PSNR metrics.

Contribution

It introduces a method that applies super resolution specifically to foreground regions in portrait images using U-Net and MR-CNN, focusing on localized enhancement.

Findings

Foreground super resolution improves image quality metrics.

Localized approach outperforms global super resolution in portraits.

Method effectively enhances main object details.

Abstract

Images play a vital role in understanding data through visual representation. It gives a clear representation of the object in context. But if this image is not clear it might not be of much use. Thus, the topic of Image Super Resolution arose and many researchers have been working towards applying Computer Vision and Deep Learning Techniques to increase the quality of images. One of the applications of Super Resolution is to increase the quality of Portrait Images. Portrait Images are images which mainly focus on capturing the essence of the main object in the frame, where the object in context is highlighted whereas the background is occluded. When performing Super Resolution the model tries to increase the overall resolution of the image. But in portrait images the foreground resolution is more important than that of the background. In this paper, the performance of a Convolutional Neural Network (CNN) architecture known as U-Net for Super Resolution combined with Mask Region Based CNN (MR-CNN) for foreground super resolution is analysed. This analysis is carried out based on Localized Super Resolution i.e. We pass the LR Images to a pre-trained Image Segmentation model (MR-CNN) and perform super resolution inference on the foreground or Segmented Images and compute the Structural Similarity Index (SSIM) and Peak Signal-to-Noise Ratio (PSNR) metrics for comparisons.