Multi-rules mining algorithm for combinatorially exploded decision trees with modified Aitchison-Aitken function-based Bayesian optimization

TL;DR

This paper introduces novel algorithms MAABO-MT and GS-MRM that efficiently construct high-performance decision trees and extract reliable, non-redundant rules, overcoming the combinatorial explosion problem in rule mining.

Contribution

The paper presents two new algorithms that strategically build decision trees and extract rules with high reliability and low computational cost, addressing the challenge of combinatorial explosion.

Findings

MAABO-MT discovers reliable rules more efficiently than random methods.

The proposed approach provides deeper insights than traditional single decision trees.

Experiments confirm the effectiveness of the algorithms on open datasets.

Abstract

Decision trees offer the benefit of easy interpretation because they allow the classification of input data based on if--then rules. However, as decision trees are constructed by an algorithm that achieves clear classification with minimum necessary rules, the trees possess the drawback of extracting only minimum rules, even when various latent rules exist in data. Approaches that construct multiple trees using randomly selected feature subsets do exist. However, the number of trees that can be constructed remains at the same scale because the number of feature subsets is a combinatorial explosion. Additionally, when multiple trees are constructed, numerous rules are generated, of which several are untrustworthy and/or highly similar. Therefore, we propose "MAABO-MT" and "GS-MRM" algorithms that strategically construct trees with high estimation performance among all possible trees with small computational complexity and extract only reliable and non-similar rules, respectively. Experiments are conducted using several open datasets to analyze the effectiveness of the proposed method. The results confirm that MAABO-MT can discover reliable rules at a lower computational cost than other methods that rely on randomness. Furthermore, the proposed method is confirmed to provide deeper insights than single decision trees commonly used in previous studies. Therefore, MAABO-MT and GS-MRM can efficiently extract rules from combinatorially exploded decision trees.