Constraint-Based Visual Generation

TL;DR

This paper introduces a novel visual generation framework that integrates deep learning with logic-based constraints, enabling more controlled and property-specific image synthesis.

Contribution

It presents a general approach combining deep learning with logic constraints, modeled as a constrained satisfaction problem, for improved visual generation.

Findings

Effective modeling of GANs and auto-encoders using the proposed logic-based approach.

Promising results in generating handwritten characters and face transformations.

Demonstrates the integration of logic constraints with deep learning enhances control in visual generation.

Abstract

In the last few years the systematic adoption of deep learning to visual generation has produced impressive results that, amongst others, definitely benefit from the massive exploration of convolutional architectures. In this paper, we propose a general approach to visual generation that combines learning capabilities with logic descriptions of the target to be generated. The process of generation is regarded as a constrained satisfaction problem, where the constraints describe a set of properties that characterize the target. Interestingly, the constraints can also involve logic variables, while all of them are converted into real-valued functions by means of the t-norm theory. We use deep architectures to model the involved variables, and propose a computational scheme where the learning process carries out a satisfaction of the constraints. We propose some examples in which the theory can naturally be used, including the modeling of GAN and auto-encoders, and report promising results in problems with the generation of handwritten characters and face transformations.