Example-Based Image Synthesis via Randomized Patch-Matching

TL;DR

This paper presents a simple, unified pyramidal algorithm for image synthesis using randomized patch-matching, specifically applied to handwritten digits and face images, with a new evaluation framework for synthesis quality.

Contribution

The paper introduces a novel pyramidal patch-matching algorithm for image synthesis and a comprehensive evaluation framework for assessing generated images.

Findings

Generated images are visually high quality and similar yet distinct from training data.

The approach effectively synthesizes handwritten digits and face images.

Evaluation shows the method produces diverse and original images.

Abstract

Image and texture synthesis is a challenging task that has long been drawing attention in the fields of image processing, graphics, and machine learning. This problem consists of modelling the desired type of images, either through training examples or via a parametric modeling, and then generating images that belong to the same statistical origin. This work addresses the image synthesis task, focusing on two specific families of images -- handwritten digits and face images. This paper offers two main contributions. First, we suggest a simple and intuitive algorithm capable of generating such images in a unified way. The proposed approach taken is pyramidal, consisting of upscaling and refining the estimated image several times. For each upscaling stage, the algorithm randomly draws small patches from a patch database, and merges these to form a coherent and novel image with high visual quality. The second contribution is a general framework for the evaluation of the generation performance, which combines three aspects: the likelihood, the originality and the spread of the synthesized images. We assess the proposed synthesis scheme and show that the results are similar in nature, and yet different from the ones found in the training set, suggesting that true synthesis effect has been obtained.