Maximum entropy methods for texture synthesis: theory and practice

TL;DR

This paper introduces a maximum entropy-based framework for texture synthesis, combining theoretical insights from information geometry and Markov chain analysis with practical experiments, including style transfer applications.

Contribution

It proposes a novel maximum entropy approach for texture synthesis, providing theoretical bounds and demonstrating effectiveness through extensive experiments and comparisons.

Findings

The method yields a maximum entropy probability measure under certain constraints.

Error bounds depend polynomially on the dimension, even in non-convex cases.

The approach outperforms some state-of-the-art methods in experiments.

Abstract

Recent years have seen the rise of convolutional neural network techniques in exemplar-based image synthesis. These methods often rely on the minimization of some variational formulation on the image space for which the minimizers are assumed to be the solutions of the synthesis problem. In this paper we investigate, both theoretically and experimentally, another framework to deal with this problem using an alternate sampling/minimization scheme. First, we use results from information geometry to assess that our method yields a probability measure which has maximum entropy under some constraints in expectation. Then, we turn to the analysis of our method and we show, using recent results from the Markov chain literature, that its error can be explicitly bounded with constants which depend polynomially in the dimension even in the non-convex setting. This includes the case where the constraints are defined via a differentiable neural network. Finally, we present an extensive experimental study of the model, including a comparison with state-of-the-art methods and an extension to style transfer.