Semi-supervised Learning with the EM Algorithm: A Comparative Study between Unstructured and Structured Prediction

TL;DR

This paper compares unstructured and structured EM-based semi-supervised learning methods, highlighting the advantages of structured approaches in robustness and incorporating structural constraints, demonstrated through flood mapping case studies.

Contribution

It provides a comparative analysis of EM-based semi-supervised learning for structured versus unstructured prediction, filling a research gap in the literature.

Findings

Structured EM is more robust to noise and class confusion.

Both methods generalize self-training with soft class assignment.

Structured method incorporates structural constraints in predictions.

Abstract

Semi-supervised learning aims to learn prediction models from both labeled and unlabeled samples. There has been extensive research in this area. Among existing work, generative mixture models with Expectation-Maximization (EM) is a popular method due to clear statistical properties. However, existing literature on EM-based semi-supervised learning largely focuses on unstructured prediction, assuming that samples are independent and identically distributed. Studies on EM-based semi-supervised approach in structured prediction is limited. This paper aims to fill the gap through a comparative study between unstructured and structured methods in EM-based semi-supervised learning. Specifically, we compare their theoretical properties and find that both methods can be considered as a generalization of self-training with soft class assignment of unlabeled samples, but the structured method additionally considers structural constraint in soft class assignment. We conducted a case study on real-world flood mapping datasets to compare the two methods. Results show that structured EM is more robust to class confusion caused by noise and obstacles in features in the context of the flood mapping application.