An Optimization Framework for Semi-Supervised and Transfer Learning using Multiple Classifiers and Clusterers

TL;DR

This paper introduces a versatile optimization framework that combines classifiers and clusterers to improve semi-supervised and transfer learning, effectively handling concept drift and enhancing classification accuracy.

Contribution

It presents a novel, scalable optimization approach that integrates multiple classifiers and cluster ensembles using Bregman divergences for improved semi-supervised and transfer learning.

Findings

Outperforms popular transductive learning methods

Effectively detects concept drift in target data

Provides a flexible framework adaptable to various loss functions

Abstract

Unsupervised models can provide supplementary soft constraints to help classify new, "target" data since similar instances in the target set are more likely to share the same class label. Such models can also help detect possible differences between training and target distributions, which is useful in applications where concept drift may take place, as in transfer learning settings. This paper describes a general optimization framework that takes as input class membership estimates from existing classifiers learnt on previously encountered "source" data, as well as a similarity matrix from a cluster ensemble operating solely on the target data to be classified, and yields a consensus labeling of the target data. This framework admits a wide range of loss functions and classification/clustering methods. It exploits properties of Bregman divergences in conjunction with Legendre duality to yield a principled and scalable approach. A variety of experiments show that the proposed framework can yield results substantially superior to those provided by popular transductive learning techniques or by naively applying classifiers learnt on the original task to the target data.