GANORCON: Are Generative Models Useful for Few-shot Segmentation?

TL;DR

This paper compares GAN-based representations and contrastive learning for few-shot segmentation, finding that contrastive learning performs comparably or better, with simpler, faster training and fewer biases.

Contribution

The study provides a systematic comparison between GAN-based and contrastive learning approaches for few-shot segmentation, highlighting the efficiency and effectiveness of contrastive methods.

Findings

GAN-based methods offer no significant performance advantage over contrastive learning.

Contrastive learning is simpler, faster, and less biased than GAN-based approaches.

Inductive biases like shape-texture disentanglement are effectively captured by contrastive models.

Abstract

Advances in generative modeling based on GANs has motivated the community to find their use beyond image generation and editing tasks. In particular, several recent works have shown that GAN representations can be re-purposed for discriminative tasks such as part segmentation, especially when training data is limited. But how do these improvements stack-up against recent advances in self-supervised learning? Motivated by this we present an alternative approach based on contrastive learning and compare their performance on standard few-shot part segmentation benchmarks. Our experiments reveal that not only do the GAN-based approach offer no significant performance advantage, their multi-step training is complex, nearly an order-of-magnitude slower, and can introduce additional bias. These experiments suggest that the inductive biases of generative models, such as their ability to disentangle shape and texture, are well captured by standard feed-forward networks trained using contrastive learning. These experiments suggest that the inductive biases present in current generative models, such as their ability to disentangle shape and texture, are well captured by standard feed-forward networks trained using contrastive learning.