Deep Low-Density Separation for Semi-Supervised Classification

TL;DR

This paper introduces a novel semi-supervised learning method that applies low-density separation to neural network embeddings, improving classification accuracy on complex, high-dimensional data.

Contribution

It proposes a new hybrid SSL approach combining low-density separation with neural network embeddings, which outperforms existing graph-based methods.

Findings

Effective classification of large unlabeled datasets

Outperforms state-of-the-art SSL methods

Suitable for high-dimensional complex data

Abstract

Given a small set of labeled data and a large set of unlabeled data, semi-supervised learning (SSL) attempts to leverage the location of the unlabeled datapoints in order to create a better classifier than could be obtained from supervised methods applied to the labeled training set alone. Effective SSL imposes structural assumptions on the data, e.g. that neighbors are more likely to share a classification or that the decision boundary lies in an area of low density. For complex and high-dimensional data, neural networks can learn feature embeddings to which traditional SSL methods can then be applied in what we call hybrid methods. Previously-developed hybrid methods iterate between refining a latent representation and performing graph-based SSL on this representation. In this paper, we introduce a novel hybrid method that instead applies low-density separation to the embedded features. We describe it in detail and discuss why low-density separation may be better suited for SSL on neural network-based embeddings than graph-based algorithms. We validate our method using in-house customer survey data and compare it to other state-of-the-art learning methods. Our approach effectively classifies thousands of unlabeled users from a relatively small number of hand-classified examples.