Structural Adversarial Objectives for Self-Supervised Representation Learning

TL;DR

This paper introduces structural adversarial objectives within GANs to enhance self-supervised representation learning, enabling discriminators to learn informative features without relying on hand-crafted data augmentations.

Contribution

It proposes novel adversarial objectives that structure features at multiple granularities, improving self-supervised learning in GANs without extensive data augmentation.

Findings

Discriminators with our objectives outperform baseline models in representation quality.

Our method achieves competitive results with contrastive learning approaches.

Effective across CIFAR-10/100 and ImageNet subset datasets.

Abstract

Within the framework of generative adversarial networks (GANs), we propose objectives that task the discriminator for self-supervised representation learning via additional structural modeling responsibilities. In combination with an efficient smoothness regularizer imposed on the network, these objectives guide the discriminator to learn to extract informative representations, while maintaining a generator capable of sampling from the domain. Specifically, our objectives encourage the discriminator to structure features at two levels of granularity: aligning distribution characteristics, such as mean and variance, at coarse scales, and grouping features into local clusters at finer scales. Operating as a feature learner within the GAN framework frees our self-supervised system from the reliance on hand-crafted data augmentation schemes that are prevalent across contrastive representation learning methods. Across CIFAR-10/100 and an ImageNet subset, experiments demonstrate that equipping GANs with our self-supervised objectives suffices to produce discriminators which, evaluated in terms of representation learning, compete with networks trained by contrastive learning approaches.