GH-Feat: Learning Versatile Generative Hierarchical Features from GANs

TL;DR

GH-Feat introduces a versatile hierarchical feature extraction method from GANs, enabling improved performance across diverse computer vision tasks including image editing, recognition, and segmentation.

Contribution

This work presents a novel encoder trained with a pretrained StyleGAN as a loss, producing transferable features that enhance various vision applications.

Findings

GH-Feat aligns well with GAN layer representations.

It improves performance in tasks like image editing and face verification.

GH-Feat enables effective semantic segmentation with limited annotations.

Abstract

Recent years witness the tremendous success of generative adversarial networks (GANs) in synthesizing photo-realistic images. GAN generator learns to compose realistic images and reproduce the real data distribution. Through that, a hierarchical visual feature with multi-level semantics spontaneously emerges. In this work we investigate that such a generative feature learned from image synthesis exhibits great potentials in solving a wide range of computer vision tasks, including both generative ones and more importantly discriminative ones. We first train an encoder by considering the pretrained StyleGAN generator as a learned loss function. The visual features produced by our encoder, termed as Generative Hierarchical Features (GH-Feat), highly align with the layer-wise GAN representations, and hence describe the input image adequately from the reconstruction perspective. Extensive experiments support the versatile transferability of GH-Feat across a range of applications, such as image editing, image processing, image harmonization, face verification, landmark detection, layout prediction, image retrieval, etc. We further show that, through a proper spatial expansion, our developed GH-Feat can also facilitate fine-grained semantic segmentation using only a few annotations. Both qualitative and quantitative results demonstrate the appealing performance of GH-Feat.