DEPARA: Deep Attribution Graph for Deep Knowledge Transferability

TL;DR

This paper introduces DEPARA, a graph-based method to evaluate the transferability of knowledge between pre-trained neural networks by analyzing attribution maps and input relatedness, aiding model and layer selection.

Contribution

The paper proposes DEPARA, a novel graph-based approach to quantify transferability between PR-DNNs, addressing model and layer selection in transfer learning.

Findings

DEPARA effectively measures transferability between models.

It outperforms existing methods in model selection tasks.

The approach is validated through extensive experiments.

Abstract

Exploring the intrinsic interconnections between the knowledge encoded in PRe-trained Deep Neural Networks (PR-DNNs) of heterogeneous tasks sheds light on their mutual transferability, and consequently enables knowledge transfer from one task to another so as to reduce the training effort of the latter. In this paper, we propose the DEeP Attribution gRAph (DEPARA) to investigate the transferability of knowledge learned from PR-DNNs. In DEPARA, nodes correspond to the inputs and are represented by their vectorized attribution maps with regards to the outputs of the PR-DNN. Edges denote the relatedness between inputs and are measured by the similarity of their features extracted from the PR-DNN. The knowledge transferability of two PR-DNNs is measured by the similarity of their corresponding DEPARAs. We apply DEPARA to two important yet under-studied problems in transfer learning: pre-trained model selection and layer selection. Extensive experiments are conducted to demonstrate the effectiveness and superiority of the proposed method in solving both these problems. Code, data and models reproducing the results in this paper are available at \url{https://github.com/zju-vipa/DEPARA}.