Eclectic Extraction of Propositional Rules from Neural Networks

TL;DR

This paper introduces HERETIC, an eclectic rule extraction method from neural networks that combines decision tree learning with network structure information, improving speed and performance for transparent AI.

Contribution

The paper presents HERETIC, a novel hybrid rule extraction algorithm that outperforms existing methods in speed and effectiveness by integrating decision trees with neural network insights.

Findings

HERETIC achieves faster rule extraction compared to existing methods.

HERETIC produces more accurate and comprehensible rules.

Theoretical analysis confirms HERETIC's efficiency and effectiveness.

Abstract

Artificial Neural Network is among the most popular algorithm for supervised learning. However, Neural Networks have a well-known drawback of being a "Black Box" learner that is not comprehensible to the Users. This lack of transparency makes it unsuitable for many high risk tasks such as medical diagnosis that requires a rational justification for making a decision. Rule Extraction methods attempt to curb this limitation by extracting comprehensible rules from a trained Network. Many such extraction algorithms have been developed over the years with their respective strengths and weaknesses. They have been broadly categorized into three types based on their approach to use internal model of the Network. Eclectic Methods are hybrid algorithms that combine the other approaches to attain more performance. In this paper, we present an Eclectic method called HERETIC. Our algorithm uses Inductive Decision Tree learning combined with information of the neural network structure for extracting logical rules. Experiments and theoretical analysis show HERETIC to be better in terms of speed and performance.