Context-dependent feature analysis with random forests

TL;DR

This paper extends the random forest framework to identify and analyze variables whose relevance depends on specific contextual information, improving understanding of complex feature interactions.

Contribution

It introduces a method to detect and characterize context-dependent variables within the random forest importance framework.

Findings

Effective identification of context-dependent variables

Application to artificial and real datasets

Enhanced understanding of feature interactions

Abstract

In many cases, feature selection is often more complicated than identifying a single subset of input variables that would together explain the output. There may be interactions that depend on contextual information, i.e., variables that reveal to be relevant only in some specific circumstances. In this setting, the contribution of this paper is to extend the random forest variable importances framework in order (i) to identify variables whose relevance is context-dependent and (ii) to characterize as precisely as possible the effect of contextual information on these variables. The usage and the relevance of our framework for highlighting context-dependent variables is illustrated on both artificial and real datasets.