An Additive Model View to Sparse Gaussian Process Classifier Design

TL;DR

This paper introduces a new additive model perspective for designing sparse Gaussian process classifiers, improving generalization and computational efficiency through adaptive basis vector selection.

Contribution

It proposes a novel stage-wise optimization method for SGPC, with new techniques for site parameter estimation and basis vector selection, enhancing performance and scalability.

Findings

Improved generalization on benchmark datasets.

Effective basis vector selection reduces computational cost.

Better performance on small basis vector sets and difficult datasets.

Abstract

We consider the problem of designing a sparse Gaussian process classifier (SGPC) that generalizes well. Viewing SGPC design as constructing an additive model like in boosting, we present an efficient and effective SGPC design method to perform a stage-wise optimization of a predictive loss function. We introduce new methods for two key components viz., site parameter estimation and basis vector selection in any SGPC design. The proposed adaptive sampling based basis vector selection method aids in achieving improved generalization performance at a reduced computational cost. This method can also be used in conjunction with any other site parameter estimation methods. It has similar computational and storage complexities as the well-known information vector machine and is suitable for large datasets. The hyperparameters can be determined by optimizing a predictive loss function. The experimental results show better generalization performance of the proposed basis vector selection method on several benchmark datasets, particularly for relatively smaller basis vector set sizes or on difficult datasets.