Nonparametric Regression and Classification with Functional, Categorical, and Mixed Covariates

TL;DR

This paper introduces a flexible nonparametric prediction method that handles mixed types of covariates, including functional and categorical data, improving accuracy and variable selection in classification and regression tasks.

Contribution

It extends the Nadaraya-Watson estimator to incorporate multiple covariate types with data-driven weighting, enhancing prediction and variable relevance assessment.

Findings

Prediction accuracy improved with the method.

Irrelevant variables can be identified and downweighted.

Effective for both classification and regression problems.

Abstract

We consider nonparametric prediction with multiple covariates, in particular categorical or functional predictors, or a mixture of both. The method proposed bases on an extension of the Nadaraya-Watson estimator where a kernel function is applied on a linear combination of distance measures each calculated on single covariates, with weights being estimated from the training data. The dependent variable can be categorical (binary or multi-class) or continuous, thus we consider both classification and regression problems. The methodology presented is illustrated and evaluated on artificial and real world data. Particularly it is observed that prediction accuracy can be increased, and irrelevant, noise variables can be identified/removed by `downgrading' the corresponding distance measures in a completely data-driven way.