Automating the Design and Development of Gradient Descent Trained Expert System Networks

TL;DR

This paper introduces a method to automate the design of gradient descent trained expert system networks, reducing manual effort and achieving low error rates, thus enhancing neural network interpretability and performance.

Contribution

It proposes a novel technique for automating the development of rule-fact networks, improving upon prior manual methods and demonstrating effective training, pruning, and re-training processes.

Findings

Error rates as low as 3.9% achieved

Networks trained and pruned effectively

Method outperforms manual development approaches

Abstract

Prior work introduced a gradient descent trained expert system that conceptually combines the learning capabilities of neural networks with the understandability and defensible logic of an expert system. This system was shown to be able to learn patterns from data and to perform decision-making at levels rivaling those reported by neural network systems. The principal limitation of the approach, though, was the necessity for the manual development of a rule-fact network (which is then trained using backpropagation). This paper proposes a technique for overcoming this significant limitation, as compared to neural networks. Specifically, this paper proposes the use of larger and denser-than-application need rule-fact networks which are trained, pruned, manually reviewed and then re-trained for use. Multiple types of networks are evaluated under multiple operating conditions and these results are presented and assessed. Based on these individual experimental condition assessments, the proposed technique is evaluated. The data presented shows that error rates as low as 3.9% (mean, 1.2% median) can be obtained, demonstrating the efficacy of this technique for many applications.