A Non-Adversarial Approach to Idempotent Generative Modelling

TL;DR

This paper introduces NAIGNs, a non-adversarial generative model that improves data reconstruction and sampling by combining IMLE with idempotent network principles, avoiding issues like mode collapse.

Contribution

The paper proposes NAIGNs, a novel non-adversarial approach that enhances idempotent generative networks using IMLE, improving stability and data distribution coverage.

Findings

NAIGNs effectively restore corrupted data.

NAIGNs generate samples closely matching the data distribution.

NAIGNs implicitly learn the data manifold's distance field.

Abstract

Idempotent Generative Networks (IGNs) are deep generative models that also function as local data manifold projectors, mapping arbitrary inputs back onto the manifold. They are trained to act as identity operators on the data and as idempotent operators off the data manifold. However, IGNs suffer from mode collapse, mode dropping, and training instability due to their objectives, which contain adversarial components and can cause the model to cover the data manifold only partially -- an issue shared with generative adversarial networks. We introduce Non-Adversarial Idempotent Generative Networks (NAIGNs) to address these issues. Our loss function combines reconstruction with the non-adversarial generative objective of Implicit Maximum Likelihood Estimation (IMLE). This improves on IGN's ability to restore corrupted data and generate new samples that closely match the data distribution. We moreover demonstrate that NAIGNs implicitly learn the distance field to the data manifold, as well as an energy-based model.