Estimating Posterior Ratio for Classification: Transfer Learning from Probabilistic Perspective

TL;DR

This paper introduces a novel transfer learning method that efficiently models differences between classifiers by learning a posterior ratio, enabling effective adaptation without sharing the same parameter space.

Contribution

It proposes a new criterion for transfer learning based on posterior ratio estimation that simplifies adaptation by focusing on differences rather than entire models.

Findings

Posterior ratio minimizes an upper bound of target risk.

Model can be trained efficiently without sharing parameter space.

Improves transfer learning by focusing on classifier differences.

Abstract

Transfer learning assumes classifiers of similar tasks share certain parameter structures. Unfortunately, modern classifiers uses sophisticated feature representations with huge parameter spaces which lead to costly transfer. Under the impression that changes from one classifier to another should be ``simple'', an efficient transfer learning criteria that only learns the ``differences'' is proposed in this paper. We train a \emph{posterior ratio} which turns out to minimizes the upper-bound of the target learning risk. The model of posterior ratio does not have to share the same parameter space with the source classifier at all so it can be easily modelled and efficiently trained. The resulting classifier therefore is obtained by simply multiplying the existing probabilistic-classifier with the learned posterior ratio.