Semi-Supervised Learning for Sparsely-Labeled Sequential Data: Application to Healthcare Video Processing

TL;DR

This paper introduces a semi-supervised learning approach for event detection in sequential data with sparse labels, demonstrating improved performance on video datasets and a real-world epilepsy application.

Contribution

The paper proposes a novel semi-supervised training strategy that leverages noisy end time estimates to enhance event detection in sparsely labeled sequential data.

Findings

Neural networks improve detection with more data despite label noise.

Risk-tolerant strategy outperforms conservative estimates by up to 30 points in mean average precision.

Method achieves comparable performance to fully supervised models in epilepsy video analysis.

Abstract

Labeled data is a critical resource for training and evaluating machine learning models. However, many real-life datasets are only partially labeled. We propose a semi-supervised machine learning training strategy to improve event detection performance on sequential data, such as video recordings, when only sparse labels are available, such as event start times without their corresponding end times. Our method uses noisy guesses of the events' end times to train event detection models. Depending on how conservative these guesses are, mislabeled samples may be introduced into the training set. We further propose a mathematical model for explaining and estimating the evolution of the classification performance for increasingly noisier end time estimates. We show that neural networks can improve their detection performance by leveraging more training data with less conservative approximations despite the higher proportion of incorrect labels. We adapt sequential versions of CIFAR-10 and MNIST, and use the Berkeley MHAD and HMBD51 video datasets to empirically evaluate our method, and find that our risk-tolerant strategy outperforms conservative estimates by 3.5 points of mean average precision for CIFAR, 30 points for MNIST, 3 points for MHAD, and 14 points for HMBD51. Then, we leverage the proposed training strategy to tackle a real-life application: processing continuous video recordings of epilepsy patients, and show that our method outperforms baseline labeling methods by 17 points of average precision, and reaches a classification performance similar to that of fully supervised models. We share part of the code for this article.