CR-GAN: Learning Complete Representations for Multi-view Generation

TL;DR

CR-GAN introduces a dual-pathway framework that enhances multi-view image generation from a single view by learning complete representations, leveraging both labeled and unlabeled data for improved generalization and realism.

Contribution

The paper proposes CR-GAN, a novel two-pathway GAN architecture that maintains complete embeddings and enables self-supervised learning for better multi-view generation.

Findings

CR-GAN outperforms state-of-the-art methods on multi-view generation tasks.

The dual-pathway design improves generalization to unseen data.

Self-supervised learning enriches the embedding space for realistic outputs.

Abstract

Generating multi-view images from a single-view input is an essential yet challenging problem. It has broad applications in vision, graphics, and robotics. Our study indicates that the widely-used generative adversarial network (GAN) may learn "incomplete" representations due to the single-pathway framework: an encoder-decoder network followed by a discriminator network. We propose CR-GAN to address this problem. In addition to the single reconstruction path, we introduce a generation sideway to maintain the completeness of the learned embedding space. The two learning pathways collaborate and compete in a parameter-sharing manner, yielding considerably improved generalization ability to "unseen" dataset. More importantly, the two-pathway framework makes it possible to combine both labeled and unlabeled data for self-supervised learning, which further enriches the embedding space for realistic generations. The experimental results prove that CR-GAN significantly outperforms state-of-the-art methods, especially when generating from "unseen" inputs in wild conditions.