Feature Importance Guided Random Forest Learning with Simulated Annealing Based Hyperparameter Tuning

TL;DR

This paper proposes a new Random Forest framework that uses feature importance guidance and simulated annealing for hyperparameter tuning, leading to improved accuracy and better feature relevance across various domains.

Contribution

It introduces a novel integration of importance-guided feature sampling with simulated annealing for hyperparameter optimization in Random Forests, enhancing performance and interpretability.

Findings

Consistent accuracy improvements across multiple datasets

Enhanced feature relevance identification

Effective hyperparameter tuning with simulated annealing

Abstract

This paper introduces a novel framework for enhancing Random Forest classifiers by integrating probabilistic feature sampling and hyperparameter tuning via Simulated Annealing. The proposed framework exhibits substantial advancements in predictive accuracy and generalization, adeptly tackling the multifaceted challenges of robust classification across diverse domains, including credit risk evaluation, anomaly detection in IoT ecosystems, early-stage medical diagnostics, and high-dimensional biological data analysis. To overcome the limitations of conventional Random Forests, we present an approach that places stronger emphasis on capturing the most relevant signals from data while enabling adaptive hyperparameter configuration. The model is guided towards features that contribute more meaningfully to classification and optimizing this with dynamic parameter tuning. The results demonstrate consistent accuracy improvements and meaningful insights into feature relevance, showcasing the efficacy of combining importance aware sampling and metaheuristic optimization.