DT-LET: Deep Transfer Learning by Exploring where to Transfer

TL;DR

This paper introduces DT-LET, a novel deep transfer learning model that determines optimal layer transfer points between heterogeneous domains with different data resolutions, improving transfer accuracy.

Contribution

It proposes a new mathematical model and loss function to identify the best layer correspondence for transfer learning across heterogeneous domains.

Findings

DT-LET outperforms existing methods in cross-domain recognition tasks.

Layer correspondence searching is crucial for effective transfer learning.

The model demonstrates significant accuracy improvements in heterogeneous data scenarios.

Abstract

Previous transfer learning methods based on deep network assume the knowledge should be transferred between the same hidden layers of the source domain and the target domains. This assumption doesn't always hold true, especially when the data from the two domains are heterogeneous with different resolutions. In such case, the most suitable numbers of layers for the source domain data and the target domain data would differ. As a result, the high level knowledge from the source domain would be transferred to the wrong layer of target domain. Based on this observation, "where to transfer" proposed in this paper should be a novel research frontier. We propose a new mathematic model named DT-LET to solve this heterogeneous transfer learning problem. In order to select the best matching of layers to transfer knowledge, we define specific loss function to estimate the corresponding relationship between high-level features of data in the source domain and the target domain. To verify this proposed cross-layer model, experiments for two cross-domain recognition/classification tasks are conducted, and the achieved superior results demonstrate the necessity of layer correspondence searching.