Cultivating Archipelago of Forests: Evolving Robust Decision Trees through Island Coevolution

TL;DR

This paper introduces ICoEvoRDF, an island-based coevolutionary algorithm that enhances decision tree ensemble robustness against adversarial attacks by fostering diversity and utilizing game theory for ensemble weighting.

Contribution

It presents a novel coevolutionary framework with island migration and Nash equilibrium-based ensemble weighting to improve decision tree robustness and accuracy.

Findings

ICoEvoRDF outperforms state-of-the-art methods on 20 datasets.

The approach improves adversarial accuracy and minimax regret.

Flexible framework integrates diverse decision tree methods.

Abstract

Decision trees are widely used in machine learning due to their simplicity and interpretability, but they often lack robustness to adversarial attacks and data perturbations. The paper proposes a novel island-based coevolutionary algorithm (ICoEvoRDF) for constructing robust decision tree ensembles. The algorithm operates on multiple islands, each containing populations of decision trees and adversarial perturbations. The populations on each island evolve independently, with periodic migration of top-performing decision trees between islands. This approach fosters diversity and enhances the exploration of the solution space, leading to more robust and accurate decision tree ensembles. ICoEvoRDF utilizes a popular game theory concept of mixed Nash equilibrium for ensemble weighting, which further leads to improvement in results. ICoEvoRDF is evaluated on 20 benchmark datasets, demonstrating its superior performance compared to state-of-the-art methods in optimizing both adversarial accuracy and minimax regret. The flexibility of ICoEvoRDF allows for the integration of decision trees from various existing methods, providing a unified framework for combining diverse solutions. Our approach offers a promising direction for developing robust and interpretable machine learning models