A Novel Splitting Criterion Inspired by Geometric Mean Metric Learning for Decision Tree

TL;DR

This paper introduces a new splitting criterion based on Geometric Mean Metric Learning to accelerate decision tree growth, achieving comparable or better accuracy with significantly reduced computation time.

Contribution

The paper proposes a novel GMML-inspired splitting criterion for decision trees that speeds up growth and simplifies extension to multivariable trees.

Findings

dGMML-DT achieves 10x speedup over traditional univariate decision trees.

The method maintains or improves classification accuracy.

Extension to multivariable trees is straightforward and efficient.

Abstract

Decision tree (DT) attracts persistent research attention due to its impressive empirical performance and interpretability in numerous applications. However, the growth of traditional yet widely-used univariate decision trees (UDTs) is quite time-consuming as they need to traverse all the features to find the splitting value with the maximal reduction of the impurity at each internal node. In this paper, we newly design a splitting criterion to speed up the growth. The criterion is induced from Geometric Mean Metric Learning (GMML) and then optimized under its diagonalized metric matrix constraint, consequently, a closed-form rank of feature discriminant abilities can at once be obtained and the top 1 feature at each node used to grow an intent DT (called as dGMML-DT, where d is an abbreviation for diagonalization). We evaluated the performance of the proposed methods and their corresponding ensembles on benchmark datasets. The experiment shows that dGMML-DT achieves comparable or better classification results more efficiently than the UDTs with 10x average speedup. Furthermore, dGMML-DT can straightforwardly be extended to its multivariable counterpart (dGMML-MDT) without needing laborious operations.