Self-Taught Support Vector Machine

TL;DR

This paper introduces a novel self-taught learning approach that leverages both labeled target data and unlabeled source data from different distributions to improve classification performance using a new shared feature space.

Contribution

It proposes a new objective function that learns a common space where conditional distributions are aligned and incorporates hidden labels of source data for robust SVM classification.

Findings

Superior performance on Caltech-256 dataset

Outperforms existing algorithms on benchmark datasets

Effective handling of different source and target distributions

Abstract

In this paper, a new approach for classification of target task using limited labeled target data as well as enormous unlabeled source data is proposed which is called self-taught learning. The target and source data can be drawn from different distributions. In the previous approaches, covariate shift assumption is considered where the marginal distributions p(x) change over domains and the conditional distributions p(y|x) remain the same. In our approach, we propose a new objective function which simultaneously learns a common space T(.) where the conditional distributions over domains p(T(x)|y) remain the same and learns robust SVM classifiers for target task using both source and target data in the new representation. Hence, in the proposed objective function, the hidden label of the source data is also incorporated. We applied the proposed approach on Caltech-256, MSRC+LMO datasets and compared the performance of our algorithm to the available competing methods. Our method has a superior performance to the successful existing algorithms.