On the use of reproducing kernel Hilbert spaces in functional classification

TL;DR

This paper explores the application of Reproducing Kernel Hilbert Spaces to Gaussian process classification, providing explicit formulas, interpreting near-perfect classification phenomena, and proposing a new variable selection method.

Contribution

It offers explicit Bayes rule expressions, interprets near-perfect classification via RKHS and mutual singularity, and introduces a model-based variable selection approach for Gaussian process classification.

Findings

Explicit formulas for optimal classification rules in Gaussian processes.

Interpretation of near-perfect classification through mutual singularity.

A new variable selection method based on RKHS structure.

Abstract

The H\'ajek-Feldman dichotomy establishes that two Gaussian measures are either mutually absolutely continuous with respect to each other (and hence there is a Radon-Nikodym density for each measure with respect to the other one) or mutually singular. Unlike the case of finite dimensional Gaussian measures, there are non-trivial examples of both situations when dealing with Gaussian stochastic processes. This paper provides: (a) Explicit expressions for the optimal (Bayes) rule and the minimal classification error probability in several relevant problems of supervised binary classification of mutually absolutely continuous Gaussian processes. The approach relies on some classical results in the theory of Reproducing Kernel Hilbert Spaces (RKHS). (b) An interpretation, in terms of mutual singularity, for the "near perfect classification" phenomenon described by Delaigle and Hall (2012). We show that the asymptotically optimal rule proposed by these authors can be identified with the sequence of optimal rules for an approximating sequence of classification problems in the absolutely continuous case. (c) A new model-based method for variable selection in binary classification problems, which arises in a very natural way from the explicit knowledge of the RN-derivatives and the underlying RKHS structure. Different classifiers might be used from the selected variables. In particular, the classical, linear finite-dimensional Fisher rule turns out to be consistent under some standard conditions on the underlying functional model.