Incremental Structure Discovery of Classification via Sequential Monte Carlo

TL;DR

This paper introduces a novel method combining Gaussian Processes and Sequential Monte Carlo to automatically discover and adapt classification models with minimal prior knowledge, suitable for online data processing.

Contribution

It extends GP-based structure discovery with SMC to handle latent variables and adapt models dynamically as new data arrives, improving online classification performance.

Findings

Automatically discovers classification structures from data.

Outperforms existing methods with up to 10% accuracy gain.

Effectively learns features on both synthetic and real-world data.

Abstract

Gaussian Processes (GPs) provide a powerful framework for making predictions and understanding uncertainty for classification with kernels and Bayesian non-parametric learning. Building such models typically requires strong prior knowledge to define preselect kernels, which could be ineffective for online applications of classification that sequentially process data because features of data may shift during the process. To alleviate the requirement of prior knowledge used in GPs and learn new features from data that arrive successively, this paper presents a novel method to automatically discover models of classification on complex data with little prior knowledge. Our method adapts a recently proposed technique for GP-based time-series structure discovery, which integrates GPs and Sequential Monte Carlo (SMC). We extend the technique to handle extra latent variables in GP classification, such that our method can effectively and adaptively learn a-priori unknown structures of classification from continuous input. In addition, our method adapts new batch of data with updated structures of models. Our experiments show that our method is able to automatically incorporate various features of kernels on synthesized data and real-world data for classification. In the experiments of real-world data, our method outperforms various classification methods on both online and offline setting achieving a 10\% accuracy improvement on one benchmark.