Forest Floor Visualizations of Random Forests

TL;DR

This paper introduces the forest floor visualization method for interpreting random forests, enabling detailed insights into feature interactions and model structure beyond traditional partial dependence plots.

Contribution

The paper presents a novel visualization technique, forest floor, that improves interpretability of random forests by revealing interactions and local details through feature contributions and projections.

Findings

Forest floor visualizations reveal interactions not seen in partial dependence plots.

The method allows zooming into local model details.

A new goodness-of-visualization measure is introduced.

Abstract

We propose a novel methodology, forest floor, to visualize and interpret random forest (RF) models. RF is a popular and useful tool for non-linear multi-variate classification and regression, which yields a good trade-off between robustness (low variance) and adaptiveness (low bias). Direct interpretation of a RF model is difficult, as the explicit ensemble model of hundreds of deep trees is complex. Nonetheless, it is possible to visualize a RF model fit by its mapping from feature space to prediction space. Hereby the user is first presented with the overall geometrical shape of the model structure, and when needed one can zoom in on local details. Dimensional reduction by projection is used to visualize high dimensional shapes. The traditional method to visualize RF model structure, partial dependence plots, achieve this by averaging multiple parallel projections. We suggest to first use feature contributions, a method to decompose trees by splitting features, and then subsequently perform projections. The advantages of forest floor over partial dependence plots is that interactions are not masked by averaging. As a consequence, it is possible to locate interactions, which are not visualized in a given projection. Furthermore, we introduce: a goodness-of-visualization measure, use of colour gradients to identify interactions and an out-of-bag cross validated variant of feature contributions.