Variational Inference for Semiparametric Bayesian Novelty Detection in Large Datasets

TL;DR

This paper introduces a scalable variational inference method for a Bayesian semiparametric novelty detection model, improving efficiency and accuracy in large, high-dimensional datasets.

Contribution

It develops a variational Bayes algorithm for the Brand model, enabling efficient novelty detection in large datasets with high-dimensional data.

Findings

Significant efficiency gains over MCMC methods

High classification accuracy demonstrated in simulations

Effective detection of novel classes in satellite data

Abstract

After being trained on a fully-labeled training set, where the observations are grouped into a certain number of known classes, novelty detection methods aim to classify the instances of an unlabeled test set while allowing for the presence of previously unseen classes. These models are valuable in many areas, ranging from social network and food adulteration analyses to biology, where an evolving population may be present. In this paper, we focus on a two-stage Bayesian semiparametric novelty detector, also known as Brand, recently introduced in the literature. Leveraging on a model-based mixture representation, Brand allows clustering the test observations into known training terms or a single novelty term. Furthermore, the novelty term is modeled with a Dirichlet Process mixture model to flexibly capture any departure from the known patterns. Brand was originally estimated using MCMC schemes, which are prohibitively costly when applied to high-dimensional data. To scale up Brand applicability to large datasets, we propose to resort to a variational Bayes approach, providing an efficient algorithm for posterior approximation. We demonstrate a significant gain in efficiency and excellent classification performance with thorough simulation studies. Finally, to showcase its applicability, we perform a novelty detection analysis using the openly-available Statlog dataset, a large collection of satellite imaging spectra, to search for novel soil types.