A Projection Pursuit Forest Algorithm for Supervised Classification

TL;DR

This paper introduces the projection pursuit random forest (PPF), an ensemble classification method that uses linear combinations of variables and projection pursuit to improve class separation, especially in multi-variable scenarios.

Contribution

It presents a novel ensemble learning algorithm, PPF, that leverages projection pursuit and linear combinations of variables for enhanced classification performance.

Findings

PPF outperforms traditional random forests in certain classification tasks.

The method is applicable to multi-class problems.

PPF is implemented in an R package available on CRAN.

Abstract

This paper presents a new ensemble learning method for classification problems called projection pursuit random forest (PPF). PPF uses the PPtree algorithm introduced in Lee et al. (2013). In PPF, trees are constructed by splitting on linear combinations of randomly chosen variables. Projection pursuit is used to choose a projection of the variables that best separates the classes. Utilizing linear combinations of variables to separate classes takes the correlation between variables into account which allows PPF to outperform a traditional random forest when separations between groups occurs in combinations of variables. The method presented here can be used in multi-class problems and is implemented into an R (R Core Team, 2018) package, PPforest, which is available on CRAN, with development versions at https://github.com/natydasilva/PPforest.