CoPE: Conditional image generation using Polynomial Expansions

TL;DR

CoPE introduces a polynomial expansion framework for conditional image generation, effectively handling two-variable inputs and capturing their correlations, demonstrated across multiple tasks and datasets.

Contribution

This work presents CoPE, a novel polynomial expansion method for conditional generation that extends PNNs to handle multiple input variables and their interactions.

Findings

Effective in class-conditional generation tasks

Performs well in inverse problems and image translation tasks

Demonstrates versatility across diverse datasets

Abstract

Generative modeling has evolved to a notable field of machine learning. Deep polynomial neural networks (PNNs) have demonstrated impressive results in unsupervised image generation, where the task is to map an input vector (i.e., noise) to a synthesized image. However, the success of PNNs has not been replicated in conditional generation tasks, such as super-resolution. Existing PNNs focus on single-variable polynomial expansions which do not fare well to two-variable inputs, i.e., the noise variable and the conditional variable. In this work, we introduce a general framework, called CoPE, that enables a polynomial expansion of two input variables and captures their auto- and cross-correlations. We exhibit how CoPE can be trivially augmented to accept an arbitrary number of input variables. CoPE is evaluated in five tasks (class-conditional generation, inverse problems, edges-to-image translation, image-to-image translation, attribute-guided generation) involving eight datasets. The thorough evaluation suggests that CoPE can be useful for tackling diverse conditional generation tasks. The source code of CoPE is available at \url{https://github.com/grigorisg9gr/polynomial_nets_for_conditional_generation}.