Generation of Explicit Knowledge from Empirical Data through Pruning of Trainable Neural Networks

TL;DR

This paper introduces a flexible, generalized method for extracting explicit knowledge from data by pruning neural networks to their minimal form, enabling rule extraction in customizable formats.

Contribution

It presents a novel three-step technology for pruning neural networks to extract explicit knowledge, allowing flexible and customizable rule extraction from empirical data.

Findings

Achieved significant reduction in network complexity.

Enabled extraction of rules in various desired formats.

Developed over 10 years, applied in numerous expert systems.

Abstract

This paper presents a generalized technology of extraction of explicit knowledge from data. The main ideas are 1) maximal reduction of network complexity (not only removal of neurons or synapses, but removal all the unnecessary elements and signals and reduction of the complexity of elements), 2) using of adjustable and flexible pruning process (the pruning sequence shouldn't be predetermined - the user should have a possibility to prune network on his own way in order to achieve a desired network structure for the purpose of extraction of rules of desired type and form), and 3) extraction of rules not in predetermined but any desired form. Some considerations and notes about network architecture and training process and applicability of currently developed pruning techniques and rule extraction algorithms are discussed. This technology, being developed by us for more than 10 years, allowed us to create dozens of knowledge-based expert systems. In this paper we present a generalized three-step technology of extraction of explicit knowledge from empirical data.