Macrocanonical Models for Texture Synthesis

TL;DR

This paper explores macrocanonical models for texture synthesis, extending Gibbs measure frameworks to real-valued images, and analyzes algorithms combining sampling and minimization with neural network features.

Contribution

It extends macrocanonical models to real-valued images and analyzes an alternating sampling-minimization algorithm for texture synthesis.

Findings

Algorithm effectively combines sampling and minimization.

Neural network features improve texture synthesis quality.

Analysis reveals advantages and drawbacks of the proposed sampling scheme.

Abstract

In this article we consider macrocanonical models for texture synthesis. In these models samples are generated given an input texture image and a set of features which should be matched in expectation. It is known that if the images are quantized, macrocanonical models are given by Gibbs measures, using the maximum entropy principle. We study conditions under which this result extends to real-valued images. If these conditions hold, finding a macrocanonical model amounts to minimizing a convex function and sampling from an associated Gibbs measure. We analyze an algorithm which alternates between sampling and minimizing. We present experiments with neural network features and study the drawbacks and advantages of using this sampling scheme.