PPCD-GAN: Progressive Pruning and Class-Aware Distillation for Large-Scale Conditional GANs Compression

TL;DR

This paper introduces PPCD-GAN, a novel method for compressing large-scale conditional GANs by progressive pruning and class-aware distillation, resulting in significantly fewer parameters while maintaining high performance.

Contribution

The paper proposes a new progressive pruning residual block and class-aware distillation technique for end-to-end GAN compression, outperforming existing methods on ImageNet.

Findings

Reduces up to 5.2x parameters on ImageNet 128x128

Maintains better performance compared to state-of-the-art methods

Enables lighter GAN models without performance loss

Abstract

We push forward neural network compression research by exploiting a novel challenging task of large-scale conditional generative adversarial networks (GANs) compression. To this end, we propose a gradually shrinking GAN (PPCD-GAN) by introducing progressive pruning residual block (PP-Res) and class-aware distillation. The PP-Res is an extension of the conventional residual block where each convolutional layer is followed by a learnable mask layer to progressively prune network parameters as training proceeds. The class-aware distillation, on the other hand, enhances the stability of training by transferring immense knowledge from a well-trained teacher model through instructive attention maps. We train the pruning and distillation processes simultaneously on a well-known GAN architecture in an end-to-end manner. After training, all redundant parameters as well as the mask layers are discarded, yielding a lighter network while retaining the performance. We comprehensively illustrate, on ImageNet 128x128 dataset, PPCD-GAN reduces up to 5.2x (81%) parameters against state-of-the-arts while keeping better performance.