Deep Photo Cropper and Enhancer

TL;DR

This paper presents a novel approach combining deep networks for cropping embedded images within photos and enhancing their quality through super-resolution, supported by a new dataset and comprehensive evaluations.

Contribution

It introduces a combined deep photo cropper and enhancer framework, utilizing spatial transformers and super-resolution, along with a new dataset for training and testing.

Findings

Effective cropping of embedded images demonstrated.

Enhanced image quality with super-resolution.

Quantitative and qualitative evaluation shows promising results.

Abstract

This paper introduces a new type of image enhancement problem. Compared to traditional image enhancement methods, which mostly deal with pixel-wise modifications of a given photo, our proposed task is to crop an image which is embedded within a photo and enhance the quality of the cropped image. We split our proposed approach into two deep networks: deep photo cropper and deep image enhancer. In the photo cropper network, we employ a spatial transformer to extract the embedded image. In the photo enhancer, we employ super-resolution to increase the number of pixels in the embedded image and reduce the effect of stretching and distortion of pixels. We use cosine distance loss between image features and ground truth for the cropper and the mean square loss for the enhancer. Furthermore, we propose a new dataset to train and test the proposed method. Finally, we analyze the proposed method with respect to qualitative and quantitative evaluations.