Optimising the attribute order in Fuzzy Rough Rule Induction

TL;DR

This paper investigates whether attribute order optimization improves fuzzy rough rule induction performance, finding that attribute removal via fuzzy rough feature selection enhances accuracy and rule simplicity, unlike order optimization.

Contribution

It demonstrates that attribute order optimization does not improve performance, but attribute removal through fuzzy rough feature selection does, advancing interpretability in rule induction.

Findings

Attribute order optimization does not improve FRRI performance.

Removing a small number of attributes improves accuracy and rule simplicity.

Fuzzy rough feature selection benefits rule induction interpretability.

Abstract

Interpretability is the next pivotal frontier in machine learning research. In the pursuit of glass box models - as opposed to black box models, like random forests or neural networks - rule induction algorithms are a logical and promising avenue, as the rules can easily be understood by humans. In our previous work, we introduced FRRI, a novel rule induction algorithm based on fuzzy rough set theory. We demonstrated experimentally that FRRI outperformed other rule induction methods with regards to accuracy and number of rules. FRRI leverages a fuzzy indiscernibility relation to partition the data space into fuzzy granules, which are then combined into a minimal covering set of rules. This indiscernibility relation is constructed by removing attributes from rules in a greedy way. This raises the question: does the order of the attributes matter? In this paper, we show that optimising only the order of attributes using known methods from fuzzy rough set theory and classical machine learning does not improve the performance of FRRI on multiple metrics. However, removing a small number of attributes using fuzzy rough feature selection during this step positively affects balanced accuracy and the average rule length.