Transferring Knowledge with Attention Distillation for Multi-Domain Image-to-Image Translation

TL;DR

This paper introduces a novel method using gradient-based attention distillation in a teacher-student framework to enhance multi-domain image-to-image translation, including cross-domain scenarios, with demonstrated improvements on facial attribute transfer and expression synthesis.

Contribution

It proposes a new attention distillation technique for GANs that leverages gradient-based explanations to improve multi-domain image translation performance.

Findings

Improved translation quality in multi-domain facial attribute transfer.

Effective knowledge transfer across related domains using pseudo-attentions.

Quantitative and qualitative validation on facial expression synthesis.

Abstract

Gradient-based attention modeling has been used widely as a way to visualize and understand convolutional neural networks. However, exploiting these visual explanations during the training of generative adversarial networks (GANs) is an unexplored area in computer vision research. Indeed, we argue that this kind of information can be used to influence GANs training in a positive way. For this reason, in this paper, it is shown how gradient based attentions can be used as knowledge to be conveyed in a teacher-student paradigm for multi-domain image-to-image translation tasks in order to improve the results of the student architecture. Further, it is demonstrated how "pseudo"-attentions can also be employed during training when teacher and student networks are trained on different domains which share some similarities. The approach is validated on multi-domain facial attributes transfer and human expression synthesis showing both qualitative and quantitative results.