Parametrizing filters of a CNN with a GAN

TL;DR

This paper introduces a method to parametrize CNN filters using a GAN's latent space, enabling the modeling of complex, high-level invariances beyond simple transformations like translation or rotation.

Contribution

The paper presents a novel approach to encode CNN filters with a GAN's latent space, capturing richer and non-linear invariances in data.

Findings

Method effectively captures complex invariances

Visualizations demonstrate non-linear invariance modeling

Enables richer invariance analysis in CNNs

Abstract

It is commonly agreed that the use of relevant invariances as a good statistical bias is important in machine-learning. However, most approaches that explicitly incorporate invariances into a model architecture only make use of very simple transformations, such as translations and rotations. Hence, there is a need for methods to model and extract richer transformations that capture much higher-level invariances. To that end, we introduce a tool allowing to parametrize the set of filters of a trained convolutional neural network with the latent space of a generative adversarial network. We then show that the method can capture highly non-linear invariances of the data by visualizing their effect in the data space.