Semi-Supervised Representation Learning based on Probabilistic Labeling

TL;DR

This paper introduces a semi-supervised representation learning algorithm that uses probabilistic labeling and HSIC to find discriminative, potentially non-linear mappings, with a performance bound to assess unlabeled data usefulness.

Contribution

The paper proposes a novel semi-supervised learning algorithm utilizing probabilistic label representations and HSIC, including a kernelized version and a performance bound for unlabeled data impact.

Findings

Effective in toy and real datasets

Kernelized version captures non-linear structures

Performance bound guides unlabeled data usage

Abstract

In this paper, we present a new algorithm for semi-supervised representation learning. In this algorithm, we first find a vector representation for the labels of the data points based on their local positions in the space. Then, we map the data to lower-dimensional space using a linear transformation such that the dependency between the transformed data and the assigned labels is maximized. In fact, we try to find a mapping that is as discriminative as possible. The approach will use Hilber-Schmidt Independence Criterion (HSIC) as the dependence measure. We also present a kernelized version of the algorithm, which allows non-linear transformations and provides more flexibility in finding the appropriate mapping. Use of unlabeled data for learning new representation is not always beneficial and there is no algorithm that can deterministically guarantee the improvement of the performance by exploiting unlabeled data. Therefore, we also propose a bound on the performance of the algorithm, which can be used to determine the effectiveness of using the unlabeled data in the algorithm. We demonstrate the ability of the algorithm in finding the transformation using both toy examples and real-world datasets.