C-Flow: Conditional Generative Flow Models for Images and 3D Point Clouds

TL;DR

C-Flow introduces a novel conditioning scheme for flow-based generative models, enabling multi-modal data modeling and applications in 3D point clouds, image manipulation, and cross-domain tasks with fine-grained control.

Contribution

The paper proposes C-Flow, a new conditioning method for normalizing flows that enhances multi-modal data modeling and introduces a strategy for unordered 3D point cloud modeling.

Findings

Effective 3D reconstruction from single images

Versatile application to image manipulation and style transfer

Demonstrated adaptability across multiple data domains

Abstract

Flow-based generative models have highly desirable properties like exact log-likelihood evaluation and exact latent-variable inference, however they are still in their infancy and have not received as much attention as alternative generative models. In this paper, we introduce C-Flow, a novel conditioning scheme that brings normalizing flows to an entirely new scenario with great possibilities for multi-modal data modeling. C-Flow is based on a parallel sequence of invertible mappings in which a source flow guides the target flow at every step, enabling fine-grained control over the generation process. We also devise a new strategy to model unordered 3D point clouds that, in combination with the conditioning scheme, makes it possible to address 3D reconstruction from a single image and its inverse problem of rendering an image given a point cloud. We demonstrate our conditioning method to be very adaptable, being also applicable to image manipulation, style transfer and multi-modal image-to-image mapping in a diversity of domains, including RGB images, segmentation maps, and edge masks.