Integrating Probabilistic Rules into Neural Networks: A Stochastic EM Learning Algorithm

TL;DR

This paper introduces a stochastic EM algorithm for probabilistic neural networks, enabling the integration of probabilistic rules and handling complex dependencies, cycles, and uncertain evidence.

Contribution

It presents a novel stochastic EM learning algorithm that combines probabilistic rules with neural networks, allowing for more flexible and expressive models.

Findings

Supports cycles of probabilistic rules

Handles hidden unobservable variables

Manages uncertain and contradictory evidence

Abstract

The EM-algorithm is a general procedure to get maximum likelihood estimates if part of the observations on the variables of a network are missing. In this paper a stochastic version of the algorithm is adapted to probabilistic neural networks describing the associative dependency of variables. These networks have a probability distribution, which is a special case of the distribution generated by probabilistic inference networks. Hence both types of networks can be combined allowing to integrate probabilistic rules as well as unspecified associations in a sound way. The resulting network may have a number of interesting features including cycles of probabilistic rules, hidden 'unobservable' variables, and uncertain and contradictory evidence.