The Generalized Proximity Forest

TL;DR

This paper introduces the Generalized Proximity Forest, extending Random Forest proximities to all supervised distance-based learning tasks, including regression and imputation, with demonstrated advantages over existing models.

Contribution

The paper presents the generalized PF model, a versatile extension of RF proximities applicable to all supervised distance-based tasks, including a new regression variant and a meta-learning framework.

Findings

Outperforms RF and k-NN models in experiments

Enables supervised imputation with pre-trained classifiers

Applicable to diverse supervised learning contexts

Abstract

Recent work has demonstrated the utility of Random Forest (RF) proximities for various supervised machine learning tasks, including outlier detection, missing data imputation, and visualization. However, the utility of the RF proximities depends upon the success of the RF model, which itself is not the ideal model in all contexts. RF proximities have recently been extended to time series by means of the distance-based Proximity Forest (PF) model, among others, affording time series analysis with the benefits of RF proximities. In this work, we introduce the generalized PF model, thereby extending RF proximities to all contexts in which supervised distance-based machine learning can occur. Additionally, we introduce a variant of the PF model for regression tasks. We also introduce the notion of using the generalized PF model as a meta-learning framework, extending supervised imputation capability to any pre-trained classifier. We experimentally demonstrate the unique advantages of the generalized PF model compared with both the RF model and the $k$-nearest neighbors model.