Random Forest Autoencoders for Guided Representation Learning

TL;DR

This paper introduces RF-AE, a neural network framework that enhances supervised data visualization by combining random forests with autoencoders, enabling scalable, out-of-sample visualization with improved accuracy and interpretability.

Contribution

RF-AE provides a scalable, out-of-sample extension for supervised visualization, integrating random forests with autoencoders to improve performance over existing methods.

Findings

RF-AE outperforms RF-PHATE in accuracy and interpretability.

RF-AE is robust to hyperparameter choices.

RF-AE generalizes to any kernel-based dimensionality reduction method.

Abstract

Extensive research has produced robust methods for unsupervised data visualization. Yet supervised visualization$\unicode{x2013}$where expert labels guide representations$\unicode{x2013}$remains underexplored, as most supervised approaches prioritize classification over visualization. Recently, RF-PHATE, a diffusion-based manifold learning method leveraging random forests and information geometry, marked significant progress in supervised visualization. However, its lack of an explicit mapping function limits scalability and its application to unseen data, posing challenges for large datasets and label-scarce scenarios. To overcome these limitations, we introduce Random Forest Autoencoders (RF-AE), a neural network-based framework for out-of-sample kernel extension that combines the flexibility of autoencoders with the supervised learning strengths of random forests and the geometry captured by RF-PHATE. RF-AE enables efficient out-of-sample supervised visualization and outperforms existing methods, including RF-PHATE's standard kernel extension, in both accuracy and interpretability. Additionally, RF-AE is robust to the choice of hyperparameters and generalizes to any kernel-based dimensionality reduction method.