Information gain ratio correction: Improving prediction with more balanced decision tree splits

TL;DR

This paper introduces an improved version of the information gain ratio for decision trees, aiming to reduce bias in split selection and enhance predictive accuracy, especially for unbalanced trees and less informative splits.

Contribution

The paper proposes an updated gain ratio that better corrects bias issues in decision tree splits compared to the original C4.5 method.

Findings

Enhanced bias correction in split selection

Improved predictive accuracy in unbalanced trees

Better handling of low-interest splits

Abstract

Decision trees algorithms use a gain function to select the best split during the tree's induction. This function is crucial to obtain trees with high predictive accuracy. Some gain functions can suffer from a bias when it compares splits of different arities. Quinlan proposed a gain ratio in C4.5's information gain function to fix this bias. In this paper, we present an updated version of the gain ratio that performs better as it tries to fix the gain ratio's bias for unbalanced trees and some splits with low predictive interest.