Multi-class Gaussian Process Classification with Noisy Inputs

TL;DR

This paper develops multi-class Gaussian process classifiers that explicitly model input noise, improving the predictive distribution's quality in noisy real-world scenarios, especially in astrophysics applications.

Contribution

It introduces novel multi-class GP classifiers that incorporate input noise using variational inference, with the ability to utilize known noise levels for enhanced performance.

Findings

Better test log-likelihood with input noise modeling

Comparable classification error across methods

Effective on synthetic, UCI, MNIST, and astrophysics data

Abstract

It is a common practice in the machine learning community to assume that the observed data are noise-free in the input attributes. Nevertheless, scenarios with input noise are common in real problems, as measurements are never perfectly accurate. If this input noise is not taken into account, a supervised machine learning method is expected to perform sub-optimally. In this paper, we focus on multi-class classification problems and use Gaussian processes (GPs) as the underlying classifier. Motivated by a data set coming from the astrophysics domain, we hypothesize that the observed data may contain noise in the inputs. Therefore, we devise several multi-class GP classifiers that can account for input noise. Such classifiers can be efficiently trained using variational inference to approximate the posterior distribution of the latent variables of the model. Moreover, in some situations, the amount of noise can be known before-hand. If this is the case, it can be readily introduced in the proposed methods. This prior information is expected to lead to better performance results. We have evaluated the proposed methods by carrying out several experiments, involving synthetic and real data. These include several data sets from the UCI repository, the MNIST data set and a data set coming from astrophysics. The results obtained show that, although the classification error is similar across methods, the predictive distribution of the proposed methods is better, in terms of the test log-likelihood, than the predictive distribution of a classifier based on GPs that ignores input noise.