Implicitly Constrained Semi-Supervised Linear Discriminant Analysis

TL;DR

This paper introduces a new semi-supervised linear discriminant analysis method based on implicit constraints, which is more robust and can outperform traditional approaches under certain conditions.

Contribution

It proposes a novel implicit constraint approach for semi-supervised LDA and analyzes its advantages over existing methods in terms of robustness and performance.

Findings

Implicit constraints improve robustness to model misspecification.

The new method can outperform traditional EM-based semi-supervised LDA.

Analysis of relationships between different semi-supervised LDA approaches.

Abstract

Semi-supervised learning is an important and active topic of research in pattern recognition. For classification using linear discriminant analysis specifically, several semi-supervised variants have been proposed. Using any one of these methods is not guaranteed to outperform the supervised classifier which does not take the additional unlabeled data into account. In this work we compare traditional Expectation Maximization type approaches for semi-supervised linear discriminant analysis with approaches based on intrinsic constraints and propose a new principled approach for semi-supervised linear discriminant analysis, using so-called implicit constraints. We explore the relationships between these methods and consider the question if and in what sense we can expect improvement in performance over the supervised procedure. The constraint based approaches are more robust to misspecification of the model, and may outperform alternatives that make more assumptions on the data, in terms of the log-likelihood of unseen objects.