Diversity Conscious Refined Random Forest

TL;DR

This paper introduces a Refined Random Forest that dynamically grows trees on informative features and enforces diversity through clustering, resulting in improved accuracy and reduced redundancy.

Contribution

It presents a novel RF variant that selectively grows trees on informative features and maximizes diversity via correlation-based clustering.

Findings

Achieves higher accuracy than standard RF on benchmark datasets.

Reduces model redundancy and inference cost.

Effectively handles both binary and multiclass classification.

Abstract

Random Forest (RF) is a widely used ensemble learning technique known for its robust classification performance across diverse domains. However, it often relies on hundreds of trees and all input features, leading to high inference cost and model redundancy. In this work, our goal is to grow trees dynamically only on informative features and then enforce maximal diversity by clustering and retaining uncorrelated trees. Therefore, we propose a Refined Random Forest Classifier that iteratively refines itself by first removing the least informative features and then analytically determines how many new trees should be grown, followed by correlation-based clustering to remove redundant trees. The classification accuracy of our model was compared against the standard RF on the same number of trees. Experiments on 8 multiple benchmark datasets, including binary and multiclass datasets, demonstrate that the proposed model achieves improved accuracy compared to standard RF.