Robust Multi-Output Learning with Highly Incomplete Data via Restricted Boltzmann Machines

TL;DR

This paper introduces a probabilistic approach using Restricted Boltzmann Machines to jointly impute missing features and labels in multi-output classification, effectively handling highly incomplete data.

Contribution

It proposes a novel joint learning framework with RBMs for incomplete multi-output data, outperforming traditional imputation-plus-classification methods.

Findings

Effective on real-world IoT security dataset

Outperforms traditional imputation methods

Handles both features and labels missing at random

Abstract

In a standard multi-output classification scenario, both features and labels of training data are partially observed. This challenging issue is widely witnessed due to sensor or database failures, crowd-sourcing and noisy communication channels in industrial data analytic services. Classic methods for handling multi-output classification with incomplete supervision information usually decompose the problem into an imputation stage that reconstructs the missing training information, and a learning stage that builds a classifier based on the imputed training set. These methods fail to fully leverage the dependencies between features and labels. In order to take full advantage of these dependencies we consider a purely probabilistic setting in which the features imputation and multi-label classification problems are jointly solved. Indeed, we show that a simple Restricted Boltzmann Machine can be trained with an adapted algorithm based on mean-field equations to efficiently solve problems of inductive and transductive learning in which both features and labels are missing at random. The effectiveness of the approach is demonstrated empirically on various datasets, with particular focus on a real-world Internet-of-Things security dataset.